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NAVTRAEQU1PCEN  75-C-0099-1 
SUMMARY 


The  following  report  describes  the  effort  and  results  of  the  application  of  a Systems 
Approach  to  Training  (SAT)  during  Phase  I of  a two-phase  R & D program.  Phase  1 was 
accomplished  under  contract  N61339-75-C-0099  and  followed  the  guidelines  established  in 
Naval  Training  Equipment  Center  Specification  N-215-254,  dated  October  1974. 

The  Phase  I effort  involved  the  design  of  an  A-6E  Pilot  and  Bombardier  (B/N)  training 
program.  The  approach  conformed  basically  to  the  Instructional  Systems  Development 
(ISD)  methodology  i.  e.  , first  analyze  the  job  and  then  design  the  training  to  fit  the  job. 

The  analysis  encompassed  the  identification  of  all  A-6E  Pilot  and  B/N  job  performance 
requirements  and  the  determination  of  those  job  performance  activities  that  must  be 
trained  in  the  Readiness  Squadron.  This  resulted  in  the  documentation  of  Pilot  and  B/N 
tasks  to  be  trained  and  task  related  factors  that  might  affect  the  priorities  assigned  within 
training  program  i.  e. , task  criticality,  frequency  of  occurrence,  inherent  difficulty, 
changes  in  knowledge  and  skill  required,  etc.  Also  identified  were  those  activities  more 
appropriately  introduced  and  practiced  in  the  operational  squadron. 

The  next  step,  designing  the  instruction,  involved;  (a)  the  determination  of  the 
criterion  objectives  and  criterion  reference  tests,  (b)  the  selection  of  instructional  methods 
and  media,  and  (c)  the  identification  and  formulation  of  instructional  units.  With  the  help 
of  a Taxonomy  of  Training  Objectives  specifically  developed  for  this  program,  the  Pilot 
and  B/N  Task  Listings  were  analyzed  to  develop  terminal  or  enabling  behavioral  statements. 
These  statements,  along  with  the  corresponding  operational  standards  and  conditions  formed 
the  substance  of  the  Criterion  Objectives.  The  criterion  reference  tests  were  determined 
by  carefully  examining  the  behavioral  statements,  the  operational  standards  and  the  opera- 
tional conditions  with  respect  to  the  training  environment  and  the  selected  media.  The 
resulting  Criterion  Test  Statements  became  the  final  element  in  the  construction  of  the 
Criterion  Objective;  also  referred  to  as  Specific  Behavioral  Objective  (SBO). 

The  final  order  of  business  was  the  identification  of  instructional  units.  Here,  as  in 
many  other  instances,  the  Navy  SMEs  were  very  helpful.  They  assisted  in  identifying 
instructional  phases  and  in  organizing  the  SBOs  into  meaningful  instructional  units  within 
these  phases.  The  SBOs  were  further  sequenced  within  these  units  and  then  tied  to  training 
media  and  preliminary  instructional  strategies  to  form  lesson  specifications.  These 
specifications  provide  the  guidelines  for  initial  learning,  practice,  and  testing  for  each 
lesson  in  the  A-6E  Pilot  and  B/N  syllabus. 

The  Phase  I study  resulted  in  the  design  of  an  instructional  system  for  A-6E  TRAM 
aircrew  training.  In  all,  over  700  Pilot  and  B/N  tasks  were  identified;  400  normal  pilot 
tasks,  approximately  200  normal  B/N  tasks  and  110  airframe  emergency  and  system 
malfunction  tasks.  More  than  370  Specific  Behavioral  Objectives  were  identified;  105  for 
the  Pilot,  109  for  the  B/N  and  159  for  both.  Generic  types  of  media  were  recommended 
to  support  Specific  Behavioral  Objectives  training  as  follows;  Audio-visual  323  SBOs, 

Visual  Aids  - 252  SBOs,  Environmental  Media  (e.  g. , WST)  - 363  SBOs,  Operational 
Equipment  (e.g.  , A/C)  - 176  SBOs,  and  Tutorial  Media  - 139  SBOs.  Lesson  Specifications 
for  55  lessons  were  developed,  with  19  designated  for  pilot  training,  17  for  B/N  training 
and  19  lessons  for  training  common  to  both  the  pilot  and  B/N.  The  following  is  a listing 
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of  the  documents  pi'epared  and  submitted  during  the  course  of  this  study.  Further 
detail  on  this  documentation  can  be  found  in  the  reference  section  of  this  report. 

a.  Work  Plan  Report,  Contract  Line  Item  No.  0001 

b.  Quarterly  Reports,  Contract  Line  Item  No.  0002 

c.  Task  Listing  A-6E  TRAM  Aircraft,  Contract  Line  Item  No,  0003 

(a)  Pilot  Procedures,  Volume  I 

(b)  B/N  Procedures,  Volume  n 

(c)  Airframe  Emergencies  and  System  Malfunctions,  Volume  III 

d.  Specific  Behavioral  Objectives  and  Criterion  Test  Statements,  Contract  Line 

Item  No.  0004 

(a)  Pilot  Procedures,  Volume  I 

(b)  B/N  Procedures,  Volume  n 

(c)  Airframe  Emergencies  and  System  Malfunctions,  Volume  HI 

e.  Training  Support  Requirements  (Media  Analysis),  Contract  Line  Item  No.  0005 

f.  Lesson  Specification  Documents  A-6E  TRAM  Aircraft,  Contract  Line  Item  No. 

0006 

g.  Final  Report,  Contract  Line  Item  No.  0007 

As  a result  of  performing  Phase  I,  sufficient  evidence  exists  indicating  that  the 
design  of  an  aircrew  training  program  may  be  more  efficiently  accomplished  through  the 
application  of  ISD  methodology.  It  also  became  evident  that  it  is  inefficient  to  employ 
operationally  oriented  personnel  in  lengthy,  highly  analytical  ISD  tasks.  They  are  more 
effectively  utilized  as  sources  of  technical  and  operational  expertise.  Based  on  the 
conclusions  reached  and  the  problems  encountered  during  this  study,  specific  recom- 
mendations were  made  and  are  contained  in  the  Recommendations  Section  of  this  report. 
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PREFACE 


This  A-6E  TRAM  ISD  Program  is  one  of  four  such  programs  currently  under  develop- 
ment for  the  Naval  aviation  community.  The  overall  project  began  in  March  of  1974,  when 
the  Naval  Training  Equipment  Center  awarded  a contract  for  the  design  and  development 
of  an  aircrew  training  program  for  the  SH-2F  LAMPS  aircraft.  This  was  followed  in 
April  with  similar  contracts  for  the  A-6E  TRAM,  EA-6B  ICAP  and  E-2C  weapons  systems. 
The  contracts  called  for  a two  phased  effort,  Phase  I of  which  has  been  completed  and  is 
described  in  this  report. 

The  research  and  development  goals  of  this  project  are  to:  (a)  evaluate  a variety  of 
ISD  methods  and  procedures  as  applied  to  several  different  aircraft  systems,  (b)  define 
the  constraints  and  operating  conditions  that  impact  aircrew  training,  and  (c)  acquire  cost, 
scheduling,  and  manpower  data  for  future  ISD  planning.  The  operational  goal  is  to  develop 
and  implement  four  viable  aircrew  training  programs. 

Special  acknowledgement  is  due  to  Captain  C.  Jasper  and  Mr.  I.  May  of  AIR-413  and 
LCDR  P.  Chatelier  of  AIR-340  for  their  assistance  and  cooperation  during  the  course  of  this 
project. 

In  addition,  special  recognition  is  due  to  VA-42,  NAS  Oceana,  which  was  designated 
model  manager  for  the  A-6E  TRAM  program  and  to  the  instructor  personnel  of  both  VA-42 
and  VA-128,  NAS  Whidbey  Island,  who  reviewed  all  interim  outputs  and  provided  in- 
valuable technical  support  throughout  the  Phase  I effort. 


JOSEPH  F.  FUNARO 
LCDR,  MSC  USN 
SCIENTIFIC  OFFICER 
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SECTION  I 
INTRODUCTION 


The  first  phase  of  a two-phase  study  which  has  as  its  purpose  the  application  of  the 
Systems  Approach  to  Training  (SAT)  in  the  A-6E  (TRAM)  aircraft  program  has  been 
completed.  Phase  I effort  involved  the  use  of  the  Instructional  Systems  Development  (ISD) 
process  through  the  design  stage  and  is  reported  herein.  Follow-on  Phase  II  effort  will 
complete  the  study  and  will  continue  the  application  of  the  ISD  process  through  the  training 
program  development  and  evaluation. 

STATEMENT  OF  THE  PROBLEM 

A review  of  naval  safety  records  over  the  past  five  or  six  years,  indicates  that  more 
than  1000  crewrmen  and  1200  aircraft  have  been  lost  through  operational  mishaps.  In  addi- 
tion to  the  immeasurable  cost  of  life,  losses  in  excess  of  two  billion  dollars  in  investments 
in  equipment  and  training  have  been  incurred.  Human  error,  often  preventable  through 
better  training,  accounted  for  a great  percentage  of  these  mishaps.  The  capabilities  of  the 
modern  weapon  system,  enabled  by  rapid  advances  in  technology,  pose  a challenge  to  the 
traditional  training  methods  to  continue  ensuring  qualified  aircrewmen. 

Until  recently,  the  general  tendency  in  aircrew  training  has  increasingly  reflected 
technological  improvements  in  designing  and  building  training  equipment.  The  training 
program  itself,  however,  tended  to  develop  in  increasing  isolation  from  the  operational 
requirements.  Training  needs  and  requirements  were  not  always  carefully  analyzed  or 
interpreted,  resulting  in  ill-defined  specifications  for  training  programs  and  equipment. 
Curricula  for  training  programs  operating  for  many  years  became  outdated  or  burdened 
by  numerous  instructional  units  appended  to  (but  not  integrated  with)  the  existing  program. 
Moreover,  these  units  tended  to  reflect  the  peculiar  and  subjective  experiences  of  the  in- 
dividual instructor  as  to  what  and  how  much  should  be  taught.  Syllabi  schedules,  and  the 
content  that  needed  to  be  covered,  tended  toward  becoming  a primary  concern  of  the  in- 
structors, rather  than  the  learning  achieved  by  the  trainee.  Training  media,  equipment, 
courses  and  lessons  received  little  follow-on  evaluation  to  ascertain  their  appropriate  use 
and  effectiveness  in  the  training  program. 

Military  leaders,  responsible  for  the  quality  of  training,  recognized  and  became 
acutely  aware  of  this  dangerous  trend  toward  isolation  of  their  training  programs  from 
their  very  "raison  d-etre,  " to  ensure  effective  performance  in  the  operational  mission. 

The  potential  consequences  of  this  trend,  if  unchecked,  are:  increased  equipment  costs 
without  proven  corresponding  increase  in  training  effectiveness,  increased  training  time 
without  verified  performance  improvement  and  tendency  towards  increased  irrelevancies 
in  the  training  program.  These  were  readily  perceived  as  intolerable. 

Alternatively,  a renewed  emphasis  was  placed  on  a systems  approach,  attempting 
to  integrate  operational  requirements  with  training  objectives,  and  training  objectives 
with  both  training  hardware  and  software  design  and  training  system  evaluation.  Appli- 
cation of  the  Instructional  Systems  Development  (ISD)  methodology  to  a number  of  air- 
crew training  programs,  both  for  aircraft  still  in  the  developmental  phase,  as  well  as 
for  some  already  operational,  has  provided  sufficient  evidence  of  training  benefits  to 
warrant  further  application,  in  this  case,  to  the  training  program  for  aircrewmen  of 
the  A-6E  TRAM  aircraft. 
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A-G  aircrew  training  is  conducted  at  VA-  12,  Oceana  and  YA-128,  Whidbey  Island. 
These  squadrons  are  in  the  process  of  accepting  A — <>  aircraft  modified  to  an  A-GE  con- 
figuration. TRAM  modifications  arc  expected  shortly.  Programs  of  training  at  these 
squadrons  are  being  conducted  at  liigh  levels  of  effectiveness  and  the  application  of  the  ISD 
process  thereto  in  no  way  reflects  dissatisfaction  with  their  accomplishments.  Because 
of  the  imminent  modifications  to  the  squadron  aircraft,  the  opportunity  exists  for  updating 
the  program  via  an  ISD  approach,  with  the  expectation  that  a good  training  program  can 
always  be  made  better. 

A second  major  problem,  always  of  concern  to  military  training  officers,  but  which 
has  become  increasingly  critical,  is  that  of  economics.  Costs  associated  with  the  deploy- 
ment of  operational  vehicles  for  training  purposes  have  been  increasing  at  alarming  rates. 
Spiraling  fuel  costs,  added  to  inflationary  increases  in  parts  replacement  and  aircraft 
maintenance,  have  driven  the  cost  of  a single  aircraft  turnaround  into  thousands  of  dollars. 
Movement  of  a group  of  A-G  pilots  and  bom  hardier /navi  gators  (B/N)  through  the  current 
Readiness  Squadron  syllabus  includes  hundreds  of  flights. 

While  acknowledging  the  efficacy  of  hands-on  training,  it  is  imperative  that  the  ex- 
tent of  use  of  operational  aircraft  in  a training  mode  be  fully  justified.  Moreover,  where 
appropriate,  less  costly  methods  of  training,  e.g. , Weapon  System  Trainers  (WST)  should 
share  the  burden  of  in-flight  training.  Application  of  ISD  technology,  with  precise  identifi- 
cation of  performance  requirements,  standai’ds,  and  training  objectives,  can  help  provide 
hard  justification  for  use  of  operational  aircraft;  wliat's  more,  it  can  assess  the  trade-off 
of  using  WSTs  or  other  media  to  supplement  training  flights. 

STUDY  OBJECTIVES 

It  is  the  intent  of  this  study  to  apply  ISD-SAT  methodology  to  the  A-GE  TRAM  aircraft 
program,  with  the  expectation  of  achieving  a twofold  set  of  benefits,  operational  and  de- 
velopmental. Attainment  of  most  of  these  benefits,  as  indicated  below,  depends  upon  com- 
pletion of  Phase  II. 

OPERATIONAL  BENEFITS.  A-G  aircrew  training  is  currently  being  conducted  at  VA-42, 
NAS,  Oceana,  Virginia,  and  at  YA-128,  NAS,  Whidbey  Island,  Washington.  Introduction 
of  TRAM  aircraft  modifications  is  expected  later.  These  modifications,  identified  and 
described  in  succeeding  subsections,  will  require  some  changes  in  the  A-6  training  program. 
Application  of  the  ISD  methodology  to  the  entire  program  is  expected  to  yield  the  following 
operational  benefits: 

• Comprehensive  ground  and  flight  syllabi  for  training  of  A-GE  aircrewmen  (Phase  II) 

• Detailed  lesson  specifications  including  recommended  instructional  media  to  sup- 
port these  lessons  (Phase  I) 

• Recommended  training  program  organized  and  tailored  to  satisfy  operational  ac- 
tivities under  operational  conditions  and  to  operational  standards  of  performance 
(Phase  II) 

• Training  program,  which  by  virtue  of  its  systematic  development  and  traceability 
of  training  requirements,  enables  program  evaluation  and  content  improvement 
through  both  qualitative  and  quantitative  measurement  and  test  (Phases  I and  II) 
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• Training  program  which  provides  more  effective  utilization  of  available  manpower 
and  resources  through  early,  well-defined  training  objectives,  and  a systematic, 
goal-oriented  progression  of  activities  leading  to  those  objectives.  Superfluous  * 
and  irrelevant  training  program  materials  are  excluded  from  the  curriculum 
(Phases  I and  II). 

DEVELOPMENTAL  BENEFITS.  The  developmental  benefits  to  be  accrued  from  this  ISD 
program  apply  to  the  ISD  as  a methodology  as  well  as  directly  to  the  A-6E  aircrew  training 
program.  They  include:  B 

• ISD  approach  which  allows  determination  and  justification  of  need  for  additional 
training  hardware  (See  Section  HI,  Media  Analysis)  for  the  A-6E  TRAM  program 

• Specification  of  the  functional  characteristics  of  required  training  devices 

• Provides  additional  ISD  data  permitting  evaluation  of  the  effectiveness  of  applica- 
tion of  the  ISD  methodology  to  an  existing  training  program,  with  respect  to: 

- quality  of  the  resulting  program 

- benefits  in  terms  of  savings  in  training  hours,  costs,  or  improved  performance 

- user  acceptance 

- cost  of  implementation 
INSTRUCTIONAL  SYSTEM  DESIGN 

DEFINITION.  Instructional  System  Development  (ISD)  is  a methodology  which  is  gaining 
increasingly  widespread  attention  and  use  among  the  various  military  training  organizations 
as  well  as  within  the  civilian  sector.  It  is  defined  in  AFM  50-2  and  AFP  50-58  (page  1-2) 

ES ! 


"a  deliberate  and  orderly  process  for  planning  and  developing 
instructional  programs,  which  insures  that  personnel  are 
taught  the  knowledges,  skills  and  attitudes  essential  for  job 
performance.  ” 1 


Encompassed  within  this  definition  are  four  key  features,  both  implicit  and  explicit, 
which,  when  taken  as  a totality,  distinguish  the  ISD  approach  from  more  traditional  ap- 
proaches to  instructional  program  development.  These  features  are  not  so  much  unique  in 
kind  as  in  emphasis.  They  are: 

• Job  performance 

• Deliberate  and  orderly  approach 

• Process 

• Teaching  essentials 

1 Department  of  the  Air  Force,  Handbook  for  Designers  of  Instructional  Systems,  Vol.  1: 
Introduction,  Headquarters,  U.S.  Air  Force,  Washington,  D.C.,  Report  AFP  50-58. 
Jan.  1974. 
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First,  ISD  uses  as  its  foundation  a rather  obvious  tenet:  what  needs  to  be  trained  is 
what  needs  to  be  done.  Job  performance  in  the  field  provides  the  solid  base  upon  which  the 
ISD  process  is  begun.  Either  by  survey  of  related  occupational  areas  (where  training  is 
to  be  developed  for  a new  job)  or  by  job  analysis  (where  field  data  is  available),  this 
approach  identifies  the  critical  training  tasks,  ensuring  that  precious  training  time  is  not 
wasted  on  tasks  which  have  little  impact  on  adequate  job  performance. 

Field  performance  also  is  used  to  supply  the  criteria  or  standards  of  behavior  appro- 
priate for  the  training  situation.  Here  tendencies  to  demand  levels  of  performance  in  ex- 
cess of  those  which  are  truly  required  are  avoided.  At  the  same  time,  the  trainee,  as  well 
as  the  instructor  is  aware  of  the  specific  goals  the  trainee  is  expected  to  achieve,  allowing 
full  concentration  and  dedication  of  effort  to  relevant  activities. 

Secondly,  the  ISD  process  is  systematic  in  its  approach.  Deliberate  and  orderly 
development  is  emphasized  throughout.  Each  step  of  the  process  can  be  logically  derived 
or  related  to  the  preceding  one,  and  is  usually  documented  for  accountability.  All  assump- 
tions are  clearly  specified  to  allow  objective  evaluation  and  judgment  about  conclusions 
drawn  therefrom.  This  systematic  feature  is  highlighted  in  the  development  of  the  be- 
havioral hierarchy,  in  which  all  the  skills,  knowledges  and  attitudes  important  to  job  per- 
formance are  extracted  and  identified  from  analysis  of  the  operational  job  performance. 
This  again  allows  an  orderly,  logical  development  of  the  training  program,  teaching  what 
needs  to  be  taught,  but  only  that.  All  training  offered  must  have  a specific  training  objec- 
tive. 


The  systematic  approach  offers  an  opportunity  for  meaningful  trade-off  analyses  to  be 
performed.  Each  unit  or  grouping  of  training,  being  well-defined,  can  be  assessed  for  the 
relative  value  of  that  training  to  job  performance  against  the  costs  associated  with  achieving 
that  training,  both  in  terms  of  time  and  investment  of  resources. 

As  a process,  considerable  emphasis  is  placed  on  the  dynamic,  iterative  aspects  of 
ISD.  Each  of  the  major  activities  comprising  the  process  involves  a feedback  loop,  updat- 
ing, modifying,  evaluating  or  verifying  the  preceding  activity.  As  a process,  it  provides 
guidelines  (rather  than  strict  rules)  emphasizing  the  "how  to",  rather  than  the  "what",  in 
planning  and  developing  the  training  program.  While  precisely  identifying  what  needs  to  be 
taught  land  learned),  the  level  of  competence  that  needs  to  be  achieved,  what  it  will  cost  to 
get  there,  and  what  acceptable  alternatives  are  available  to  the  program  manager,  con- 
siderable latitude  is  allowed  in  how  to  get  there.  This  permits  training  programs  to  be  de- 
veloped with  comparable  effectiveness  and  efficiency  for  small  or  large  instructional  group 
presentations  or  individual,  self-paced,  programmed  instruction.  Furthermore,  the  use 
of  alternative  methodological  techniques,  even  within  the  same  program  or  for  the  same 
lessons,  are  often  suggested  by  the  process.  This  feature  is  particularly  welcome  to  the 
military  instructor  who  must  be  able  to  respond  to  dynamic,  often  unpredictable  changes  in 
the  training  schedule  and  situation,  to  wit,  weather,  down  aircraft  or  trainers,  personnel 
reassignments,  etc.  The  ISD  approach  thus  allows  modification  of  the  training  program  by 
changes  in  use  of  resources  rather  than  by  eliminating  or  reducing  content. 


10 


d 


NAVTRAEQUIPCEN  75-C-0099-1 

Finally,  ISD  identifies  clearly  all  the  skills,  knowledges  and  attitudes  that  need  to  be 
taught  in  order  to  ensure  adequate  performance.  This,  obviously,  does  not  include  all  the 
activities  which  comprise  the  jobs  addressed,  namely  that  of  pilot  and  B/N.  The  ISD  ap- 
proach involves  early  identification  of  tasks  for  which  skills  and  knowledges  are  already  in 
the  repertoire  of  the  individual  beginning  training,  either  because  of  prior  training  or  be- 
cause they  are  so  common  as  not  to  require  training.  It  also  identifies  tasks  for  which  only 
partial  training  need  be  attained  (further  improvement  toward  achieving  an  operational  per- 
formance standard  level  can  occur  in  next  assignment)  or  which  because  of  relatively  long 
periods  of  non-use  between  training  and  application  might  be  better  taught  immediately  prior 
to  application  (e.g. , in  operational  squadron).  All  tasks  remaining  require  training. 

Using  task-related  characteristics  or  features  such  as  complexity  of  activity,  fre- 
quency of  occurrence,  impact  on  safety  of  flight  if  performed  marginally  or  omitted,  and 
relationship  to  mission  success,  a training  requirements  analysis  is  performed.  The  re- 
sult is  a "priority"  listing  of  tasks  which  the  designer  uses  to  ensure  that  all  essential 
training  is  appropriately  planned  and  that  the  allocation  of  training  time  is  optimized. 

BACKGROUND.  ISD  is  a remarkably  eclectic  system,  having  evolved  from  advances  in  a 
number  of  disciplines  and  fields.  A considerable  body  of  knowledge  has  resulted  from  ex- 
tensive research  through  the  years  in  the  area  of  behavioral  sciences.  Learning  theories 
and  principles,  identification  and  classification  of  human  behavior,  performance  measure- 
ment and  evaluation  are  some  of  its  contributions.  From  the  educational  field  have  come 
advances  in  test  construction,  development  of  instructional  media  and  pedagogical  tech- 
niques, and  empirical  application  and  assessment  of  behavioral  principles,  among  others. 

The  field  of  management  has  added  to  ISD  development  by  providing  techniques  such  as  job 
analysis  and  occupational  surveys,  communication,  motivation  and  attitude  studies,  and 
criteria  development. 

Within  the  above  spheres  of  influence  there  are  two  interrelated  factors  which  perhaps 
more  than  any  others  can  claim  major  responsibility  for  the  refinement  and  fruition  of  the 
ISD  approach  used  today.  They  are  the  evolution  of  programmed  instruction  and  the  ex- 
plosive expansion  in  computer  technology  and  application. 

The  entrance  of  teaching  machines,  scrambled  books,  and  the  like,  upon  the  educational 
scene  in  the  late  fifties  and  early  sixties,  although  falling  short  of  its  heralding  as  a "revo- 
lutionary new  approach  to  teaching, " nevertheless  did  represent  considerable  effort  in  the 
beginnings  of  programmed  instruction.  Concepts  and  techniques  of  linear  and  branching 
programs  required  close  attention  to  the  precise,  explicit  formulation  of  the  tasks  to  be 
trained  and  training  objectives,  and  to  the  presentation  of  instructional  materials.  With 
concomitant  development  of  instructional  delivery  systems  (sound/slide,  CRT,  etc.),  in- 
dividualized, custom-tailored  training  became  feasible;  a viable  alternative  to  traditional, 
uniform,  classroom  instruction. 

As  is  often  the  case,  improvement  or  progress  in  one  area,  creates  new  or  additional 
burdens  elsewhere.  Making  use  of  this  newly  developed  flexibility  in  training  programs 
placed  enormous  demands  for  data  gathering  and  recording,  formerly  the  responsibility  of 
the  instructors,  but  which  could  now  be  readily  transferred  to  computers.  Scheduling  of 
resources  and  facilities  for  training,  monitoring  of  progress  of  students  through  the  pro- 
gram, recording  test  materials  and  results,  and  gathering  and  storing  data  to  permit  evalu- 
ation of  student  performance,  as  well  as  assessment  of  the  training  program  itself  is 
readily  and  economically  accomplished  with  the  computer. 
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With  this  development  in  techniques  for  precise  definition  of  training  objectives  and 
the  instructional  material  content  and  form  to  achieve  the  objectives,  combined  with  a capa- 
bility for  scheduling,  monitoring,  recording  and  processing,  the  ISD  approach  can  be  ap- 
plied to  any  training  situation. 

SYSTEM  DESCRIPTION 

AIRCRAFT.  The  A-6E  TRAM  aircraft  is  a two-man  subsonic,  mid-wing  attack  aircraft, 
with  side-by-side  seating  for  the  pilot  and  bombardier/navigator  (B/N).  Designed  and 
manufactured  by  Grumman  Aerospace  Corp. , Bethpage,  L.  I. , N.  Y. , A-6E  TRAM  is  a true- 
all -weather  attack  aircraft  that  can  carry  a payload  of  8.5  tons.  A wide  variety  of  weapons 
can  be  delivered  without  the  crew  ever  seeing  the  ground  or  target. 

The  aircraft  is  powered  by  two  Pratt  & Whitney  turbojet  engines  and  is  characterized 
by  a large  nose  radome  and  swept-back  wings.  It  has  arrested  landing  capabilities  and  a 
relatively  slow  approach  and  landing  speed  for  a jet  aircraft.  It  can  be  air  refueled  as  well 
as  being  used  as  a tanker.  A single-point  ground  refueling  capability  shortens  turnaround 
time  and  eases  normal  refueling.  Automatic  isolation  of  less-important  electrical  loads, 
redundant  hydraulic  power  systems,  and  tandem  actuators  minimize  the  chance  of  complete 
loss  of  essential  functions  due  to  battle  damage  or  system  malfunctions. 

TRAM  (Target  Recognition,  Attack,  Multisensor)  configured,  the  A-6E  aircraft  has 
a completely  integrated  computer  navigation  and  aircraft  control  system,  radar,  armament, 
flight  sensors,  and  cockpit  displays  which  permits  all-weather  operation  under  varying  tac- 
tical requirements . The  integration  of  these  systems  results  in  an  aircraft  with  the  ver- 
satility to  perform  not  only  as  an  attack  bomber,  but  also  in  close  support  of  ground  per- 
sonnel. 

The  major  features  of  the  A-6E  TRAM  configuration  include: 

a.  Ballistics  Computer  Set  AN/ASQ-155  — a stored  program  digital  computer  with 
increased  capability.  This  computer  is  the  center  of  the  integrated  weapons  system,  pro- 
viding flight  data,  navigation  and  control  commands,  target  tracking,  attack  and  various 
inputs  needed  for  proper  arming,  fuzing  and  release  of  selected  stores. 

b.  Carrier  Aircraft  Inertial  Navigation  System  (CAINS)  - an  increased  accuracy 
navigational  system. 

c.  Detecting  and  Ranging  Set  (DRS)  — providing  enhanced  target  identification  and 
recognition  capability  via  a forward-looking  infrared  receiver  (FLIR)  and  laser  rangefinder 
and  receiver,  DRS  control  panel  and  FLIR  display  imagery.  The  DRS  also  includes  a BIT 
test. 

d.  Armament  Control  Unit  (ACU)  — modified  to  permit  safe  arming  and  release  of 
expanded  selected  ordnance. 

e.  Modified  Communications  Navigation  and  Instrumentation  System  (CNI). 

f.  Automatic  Flight  Control  System  (AFCS)  — providing  three-axis  stability  augmen- 
tation attitude  control  and  automatic  flight  path  control  of  the  aircraft. 
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g.  All-Weather  Carrier  Landing  System  (ACLS)  - providing  automatic  touchdown 
capability  for  carrier  aircraft.  Commands  generated  by  shipboard  computers  enable  air- 
craft AFCS  commands,  providing  safe,  reliable  final  approach  and  landing  during  daylight 
or  darkness , with  minimum  interference  from  conditions  of  severe  weather  and  sea  state 
and  no  limitation  from  low  ceilings  or  visibility.  System  provides  three  modes  of  operation 
differing  in  the  extent  of  automation. 

h.  Modification  to  accept  Condor  missile  includes  control  units,  mounting,  cabling 
and  display  inputs . 

MISSION.  The  mission  of  the  A-6  Intruder  is  to  perform  high  and  low  altitude  all-weather 
attacks.  These  attacks  might  be  conducted  as  close  air  support  for  ground  forces,  or  as 
short-  or  long-range  interdiction  raids  (for  example,  against  installations  or  roads).  The 
A-6E  TRAM  aircraft,  while  not  adding  any  new  missions,  significantly  improves  weapons 
system  delivery  capability.  Because  the  A-6  is  capable  of  performing  just  about  every 
type  VFR  and  IFR  attack  mission,  its  stores  are  varied.  It  can  deliver  a large  selection 
of  conventional  and  nuclear  weapons,  bombs,  rockets,  guns  and  missiles,  as  appropriate, 
to  a multitude  of  air  and  surface  targets.  Bombs,  rockets  and  missiles  can  be  released 
in  either  a manual  or  automated  mode,  whereas  guns  are  manually  fired.  The  aircraft  is 
provided  with  a number  of  delivery  techniques,  controlled  by  the  aircrew,  or  which,  at 
crew  discretion,  can  be  fully  automated.  If  system  degradation  or  tactical  requirements 
demand,  the  aircraft  can  be  used  as  a VFR  bomber. 

Although  specific  TRAM  tactics  have  not  yet  been  developed,  missions  for  which  the 
A-6E  TRAM  aircraft  might  be  deployed  are  outlined  as  follows: 

• Fixed  land  target  attack 

• Route  reconnaissance 

• Ship  attack 

• Mine  laying 

• Trails  and  roads  interdiction 

Associated  mission  functions,  including  ECM  detection,  can  also  be  performed  by  the 
aircraft.  Air  combat  maneuvering  is  generally  limited  to  escape  techniques.  The  air- 
craft is  not  designed,  nor  equipped,  for  use  as  a fighter  aircraft.  It  has  air-to-air  missile 
defense  capability.  An  AMTI  (Airborne  Moving  Target  Indication)  feature  of  the  A-6E 
allows  detection  of  moving  targets  in  all  weather,  day  or  night,  and  attack  with  the  weapon 
system. 

Fixed  Land  Target  Attack.  Attack  missions  of  this  type  include  bridges,  buildings  or 
other  similar  installations.  Available  intelligence  information  such  as  photographs  or 
video  tapes  of  the  target  and  target  area,  maps,  location  and  type  of  target  and  expected 
defenses,  permits  appropriate  planning  for  approach  to  target,  weapon  selection  and  de- 
livery techniques  and  escape  maneuvers  . FLIR  capability  enhances  identification  of  ter- 
rain features  known  to  be  IR  sensitive,  e.g.  , water/land  boundaries,  as  well  as  target 
features,  e.g. , full  tanks  vs.  empty  in  a fuel  storage  depot.  Acquisition  of  targets  in 
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complexes  (e.g. , one  building  in  a cluster)  requires  good  area  orientation,  photos  and  other 
aids.  Attack  might  be  performed  with  a low  altitude,  terrain  following  approach.  Variety 
of  weapon  fuzing,  time  delays  and  spacing  is  available  according  to  the  particular  target. 

Route  Reconnaissance.  This  type  mission  involves  A-6  patrol  or  armed  reconnaissance  of 
a predetermined  highway,  trail,  etc.  Targets  sighted  within  this  area  of  operation  could  be 
sighted,  tracked  and  reported  on  and  an  attack  initiated  according  to  rules  of  engagement. 
Trucks,  convoys,  tanks  and  troops  are  typical  of  the  targets  to  be  encountered.  A forward 
air  controller  capability  also  allows  other  plane  or  even  ground  laser  designated  targets  to 
be  attacked. 

Ship  Attack.  Ship  attack  might  be  performed  by  a single  aircraft  or  by  an  alpha  (multiple 
aircraft)  strike.  Both  radar  and  FLIR  are  available  for  target  acquisition  and  classifica- 
tion and  heading  determination.  Laser  capability  can  be  used  to  improve  performance. 
Approach  delivery  mode  and  escape  maneuver  would  vary  according  to  ships  defense,  de- 
sign, etc. 

Mine  Laying.  Mine  laying  missions  can  be  of  three  kinds:  (1)  laying  explosive-loaded 
mines  for  combat  roles;  (2)  laying  actuation  dummies,  for  use  in  mine  sweeping  drills  and 
evaluation;  and  (3)  laying  dummies,  used  in  mine  planting  exercises. 

Placement  of  mines  is  determined  by  the  strategic  and  tactical  goals  of  the  mission. 
A-6E  aircraft  have  advantage  over  surface  and  subsurface  mine  laying  in  that  they  can  plant 
mines  in  previously  mined  areas . High  speed  and  freedom  of  maneuvering  of  the  aircraft 
also  provides  an  element  of  surprise.  Conversely,  a disadvantage  is  the  smaller  carrying 
capability  of  the  aircraft.  A number  of  onboard  systems  can  be  used  for  mine  laying. 

Trails  and  Roads  Interdiction.  This  mission  might  be  considered  a t/pe  of  fixed-target 
attack  mission,  with  a particular  strategy  of  prevention  of  use  of  the  trail  or  road  for  move- 
ment of  enemy  troops  or  equipment. 

TRAINING  PROGRAM.  The  East  Coast  A-G  Readiness  Squadron,  VA-42,  is  currently  con- 
ducting transition  training  for  four  categories  of  pilots  and  two  categories  of  bombardier/ 
navigators.  A summary  of  these  categories  and  of  the  major  training  elements  associated 
with  them  are  provided  in  Tables  1 and  2.  Both  pilot  and  B/N  training  programs  are  divided 
into  discrete  phases  consisting  of:  ground  school  activities  such  as  lectures  and  demonstra- 
tions, simulator  exercises,  and  A/C  flights.  The  major  activities  associated  with  each 
phase  of  training  for  the  category  1 (first  tour  in  A-6)  pilot  and  B/N  are  described  as  fol- 
lows. 

PILOT  PHASES 

Familiarization  Phase.  The  replacement  pilots  (RP)  are  paired  with  an  instructor  pilot 
(IP)  on  all  flights  except  two,  on  which  an  instructor  bombardier/navigator  (IBN)  may  be 
substituted.  These  flights  involve  a complete  checkout  in  the  aircraft  under  day  VFR  con- 
ditions. Included  in  these  flights  are  aerobatics,  formation  flying  and  precision  approaches 
in  addition  to  practice  landings  in  all  configurations.  Two  of  the  nine  familiarization 
flights  are  flown  at  night. 
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Visual  Weapons  Phase.  Replacement  pilots  receive  fourteen  flights . These  flights  acquaint 
the  replacement  with  manual  deliveries  associated  with  various  conventional  weapons.  Em- 
phasis is  placed  on  the  replacements  obtaining  a qualifying  dive  bomb  50-ft  CE  (circular 
error).  Three  of  the  flights  are  Close  Air  Support  (CAS)  and  simulated  ALPHA  strikes. 

The  CAs  flight  utilizes  maximum  available  live  ordnance  under  Forward  Air  Control  (FAC), 
and  the  ALPHA  strikes  simulate  current  strike  procedures  with  heavy  ordnance  loads.  All’ 
flights  proceed  to  and  from  the  targets  in  formation  utilizing  tactical  maneuvers. 


TABLE  1.  VA-42  A-6  READINESS  PILOT  TRAINING  SUMMARY(a) 


Category 

Number  of 
training  flights 

A-6  TC-4C 

Flight  hours 
A-6  TC4C 

WST 

hours 

Ground 
training 
lectures,  fa) 
hours 

Total 

syllabus, 

weeks 

(1) 

First  tour 
in  model 

65 

2 

102.5 

6.5 

12.5 

192.3 

27 

(H) 

Second  tour 
in  model 

49 

0 

69.5 

0 

12.5 

146.3 

22 

(II) 

Model  train- 
ing for  com- 
mander and 
staff  of  car- 
rier air 
group 

36 

0 

51.5 

0 

9.0 

102.8 

17 

(IV) 

Ferry 

pilots 

4 

0 

10.0 

0 

4.5 

24.0 

5 

TABLE  2.  VA-42  A-6  READINESS  BOMBARDIER/NAVIGATOR  TRAINING  SUMMARY^ 


Number  of  Ground 


C ategory 

training  flights 
A-6  TC-4C 

Flight  hours 
A-6  TC-4C 

WST 

hours 

training 
lectures,  fa) 
hours 

Total 

syllabus, 

weeks 

(I)  First  tour 

in  model 

32 

10 

61.5 

35.0 

14.5 

216.5 

27 

(II)  Second  tour 
in  model 

17 

3 

31.0 

11.0 

4.5 

140.5 

19 

(a) 

Data  is  based  on  latest  schedules  provided  by  VA-42  SMEs. 

^Ground  Training  figures  do  not  include  courses  listed  as  "Navy  Formal  School 
Training,"  e.g. , Fire  Fighting  School. 
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Navigation  Phase.  Replacement  pilots  receive  five  navigation  flights  which  include  two 
Dead- Reckoning  (DR)  flights,  two  system  navigation  flights,  and  one  TC-4C  system  naviga- 
tion flight.  During  the  DR  flights,  the  pilot  acquires  sound  dead- reckoning  techniques  and 
leams  to  identify  visual  check  points  while  flying  at  low  altitude  and  high  air  speeds.  Sys- 
tem navigation  flights  introduce  replacement  pilots  to  radar  navigation,  radar  scope  in- 
terpretation, system  navigation  equipment  capabilities  and  techniques,  and  the  terrain 
following  feature  of  the  A-6E.  The  TC-4C  flight  involves  the  replacement  pilot  operating 
the  A-6E  system,  utilizing  those  pilot's  displays  which  are  available. 

Tactics  Phase.  Replacement  pilots  are  introduced  to  and  practice  offensive  and  defensive 
air  combat  maneuvers,  and  day  and  night  air-to-air  refueling.  Replacements  learn  to  fly 
the  aircraft  to  maximum  performance  parameters  in  air  combat  maneuvering  within  the 
operational  envelope  of  the  aircraft. 

System  Weapons  Phase.  This  phase  consists  of  nine  flights  which  introduce  the  replace- 
ments to  all-weather  attack  training  and  capabilities  of  the  A-GE.  They  learn  to  interpret 
the  vertical  display  indicator  (VDI)  in  precise  detail  and  practice  combat  breakaway  maneu- 
vers from  defended  targets.  The  replacements  become  capable  of  using  all  attack  modes 
in  the  A-6E  delivery  system. 

Carrier  Qualification  Phase.  This  phase  prepares  replacements  pilots  for  initial  A-6  car- 
rier qualification,  both  day  and  night,  and  familiarizes  replacement  bombardier/navigators 
with  carrier  operations.  During  day-flights  of  field  carrier  landing  practice  (FCLP),  re- 
placement pilots  are  trained  to  fly  the  FCLP  pattern  under  simulated  night,  reduced  visibil- 
ity conditions.  Sound,  smooth  corrections  are  emphasized.  The  carrier  qualification  por- 
tion consists  of  ten,  day  arrested  landings  and  six,  night  arrested  landings  which  replace- 
ment pilots  must  satisfactorily  complete  in  order  to  qualify.  Replacement  bombardier/ 
navigators  are  introduced  to  the  day  and  night  carrier  landing  environment. 

BOMBARDIER/NAVIGATOR  PHASES 

Familiarization  Phase.  Replacement  bombardier/navigators  (RBN)  are  acquainted  with  the 
aircraft  flight  characteristics,  the  local  operating  area,  preflight  procedures,  cockpit 
procedures  and  air-to-ground  communication  procedures  used  in  the  A-6. 

Navigation  Phase.  In  the  navigation  phase,  replacement  bombardier/navigators  are  taught 
dead- reckoning  (DR)  navigation  procedures,  route  planning,  radar  and  computer  fundamen- 
tals. They  are  given  instruction  in  search  and  doppler  radar,  inertial  navigation  system, 
computer,  air  data  computer,  and  AMTI  circuitry  which  are  the  primary  components  of 
A-6  attack/navigation  system.  Replacements  fly  11  navigation  flights:  five  in  the  TC-4C, 
six  in  the  A-6. 

Radar  Target  Identification  Phase.  This  phase,  an  extension  of  the  theories  and  practices 
learned  in  the  navigation  phase,  acquaints  the  replacement  bombardier/navigator  with 
precise  target  acquisition  and  following,  terminating  in  a simulated  attack.  Replacements 
fly  three  flights  in  the  TC-4C  with  instructor  B/Ns  and  five  flights  in  the  A-6  with  instructor 
pilots.  The  flight  envelope  ranges  from  high  altitude/low  speed  (7000-ft  MSL/270  kt)  to  low 
altitude/high  speed  (1000-ft  AGL/360  kt). 
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System  Weapons  Phase,  This  phase  acquaints  replacement  bombardier/navigators  with  all 
modes  and  methods  of  delivering  conventional  and  nuclear  ordnance  using  the  A-6  weapons 
system.  Replacements  fly  15  flights  in  this  phase  which  consist  of  bombing,  mining,  AMTI, 
and  radar  target  identification  (RTI).  RTI  is  the  most  demanding  type  of  bombing  for  the 
replacement  and  requires  a maximum  of  preflight  planning.  This  phase  polishes  the  pro- 
fessional skill  of  replacement  in  utilizing  their  weapon  system  by  making  repeated  attacks 
against  ranges  and  complex  targets . 

Carrier  Qualification  Phase.  Replacement  bombardier/navigators  fly  with  replacement 
pilots  during  FCLP  sorties  and  initial  carrier  qualification  (both  day  and  night).  Although 
this  is  required  as  a part  of  the  replacement  bombardier/navigators  syllabus,  training  is 
accountable  to  the  replacement  pilot. 

Visual  Weapons  and  Tactics.  Replacement  bombardier/navigators  syllabus,  training  is 
tachment  during  their  training  to  fly  with  replacement  pilots  for  visual  bombing.  Tactics 
flights  are  flown  with  instructor  pilots . 
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SECTION  II 
OVERVIEW 


ASSUMPTIONS  AND  CONSTRAINTS 

Phase  I of  this  ISD  project  was  devoted  to  the  design  of  an  A-6E  TRAM  aircrew 
training  program.  Within  the  scope  of  this  effort  a number  of  assumptions  were  made  and 
constraints  applied  throughout  the  conduct  of  the  work.  These  are  identified  and  briefly 
discussed  below. 

a.  VA-42  was  designated  Squadron  Model  Manager  for  the  program.  All  Navy 
personnel  participation  including  Subject  Matter  Experts  (SME)  was  to  be 
coordinated  through  that  squadron. 

b.  The  training  program  being  developed  was  to  be  limited  to  the  Readiness 
Squadron  portion  of  pilot  and  B/N  training.  No  effort  was  to  be  expended  in 
determining  training  requirements,  media  recommendations,  etc. , for  basic 
flight  training  in  the  Air  Training  Command  (ATC),  operational  squadrons,  or 
with  the  combined  requirements  of  readiness  and  operational  squadrons  sharing 
common  base  facilities. 

c.  When  the  job  descriptions  were  completed,  no  attempt  was  to  be  made  by  the 
Navy/contractor  ISD  team  to  evaluate  the  appropriateness  of  the  location  at 
which  training  for  those  activities  was  being  conducted.  The  SMEs  identified, 
those  activities  which  were  taught  in  the  ATC  phase  and  which  therefore  could 
be  considered  as  Readiness  Squadron  entry  level  behavior.  They  also  indicated 
those  which  were  being  trained  in  the  Readiness  Squadron  and  those  which  wfere 
under  the  jurisdiction  of  the  operational  squadron.  Only  those  activities  for 
which  there  was  some  Readiness  Squadron  involvement  (either  being  introduced 
or  practiced  therein)  were  addressed  and  subjected  to  the  ISD  process. 

d.  The  numbers  of  pilots  and  B/N's  assigned  to  and  arriving  at  the  Readiness 
Squadron  at  any  one  time  were  such  that  a resulting  class  size  would  be  less 
than  10  or  12  individuals.  Nonvalidity  of  this  assumption  could  impact  the 
recommendations  made  for  media  utilization.  For  example,  demonstrations 
using  the  WST  might  have  been  accomplished  with  different  media  if  class  size 
was  greater  than  20. 
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e.  During  the  time  the  replacement  pilot  and  B/N  are  assigned  to  the  Readiness 
Squadron,  they  take  a number  of  formal  Navy  school  courses.  These  courses 
are  not  uniquely  part  of  A-6E  training  but  are  common  to  all  NFOs.  They 
include  courses  such  as  Intelligence  Training  and  Aviation  Physiology  Training 
(ejection  seat  and  pressure  chamber);  Water  Survival  Training;  and  Survival, 
Evasion,  Resistance  and  Escape  (SERE).  These  were  not  to  be  addressed  as 
part  of  the  ISD  process,  but  must  be  considered  when  preparing  a syllabus. 
Those  formal  courses  which  are  A-6  oriented,  (e.g.  , Naval  Air  Maintenance 
Training  and  Conventional  Weapons  School)  were  addressed. 

In  addition,  with  the  concurrence  of  the  SMEs,  all  task  activities  involving 
secret  security  classifications  (such  as  in  the  EW,  ECM  and  nuclear  areas) 
were  not  addressed.  The  attempt  was  made  to  describe  the  remaining  activ- 
ities of  the  pilot  and  B/N  in  such  a manner  as  to  permit  the  lowest  security 
classification  possible  while  still  meeting  the  requirements  of  the  process.  As 
a result,  only  the  task  listing  of  B/N  activities  and  the  treatment  of  aircraft 
tactics  required  confidential  classification.  These  are  included  in  the  same 
document1.  All  other  documentation  produced  during  Phase  I was  unclassified. 

f.  The  scope  of  this  Phase  I contract  required  the  application  of  the  ISD  process 
through  the  development  of  Lesson  Specifications,  but  excluded  the  integration 
of  the  lessons  into  a total  syllabus  of  training  for  the  replacement  pilot  and  S/N. 

g-  Four  types  of  pilot  personnel  receive  training  in  the  A-6  Readiness  Squadron 

(1)  First  tour  aviator  (first  tour  in  model) 

(2)  Second  tour  pilot  (return  from  fleet  assignment) 

(3)  Commander  (training/briefing  on  A-6E  capability) 

(4)  Ferry 

h.  Two  types  of  B/N  personnel  receive  training  in  the  A-6  Readiness  Squadron 

(1)  First  tour  B/N  (first  tour  in  model) 

(2)  Second  tour  B/N 

The  ISD  process  was  applied  to  the  design  and  development  of  a training 
program  for  the  first  tour  pilots  and  B/Ns,  the  least  knowledgeable  with  regard 
to  A-6  operations  of  all  the  forementioned  types.  This  type  airman  is  also  the 
most  representative  of  the  types  entering  the  Readiness  Squadron.  In  compari- 
son, the  first  tour  airman  brings  to  the  squadron  relatively  minimal  entry  level 
skills  and  knowledges,  and  therefore  requires  the  broadest  scope  and  most  com- 
prehensive training  program.  Other  pilot  or  B/N  types  require  some  lesser 
amounts  of  training,  which  can  be  stylized  in  accordance  with  their  particular 
skills  and  training  requirements. 

J.  Hanish,  Task  Listing,  A-6E  (TRAM)  Aircraft,  Bombardier/Navigator  Procedures, 

Vol.  II,  Grumman  Aerospace  Corp. , Bethpage,  N.Y. , Report  ISD  (SAT)  0003-2. 

Sept.  1975. 
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APPROACH 

According  to  how  they  group  and  identify  activities,  most  proponents  of  the  use  of  the 
ISD  approach  are  in  general  agreement  that  there  are  5 to  9 major  and  distinct  activities 
included  in  the  process.  Figure  1 provides  a block  diagram  representation  of  the  major 
ISD  activities  as  identified  in:  (a)  Air  Force  Manual  50-2,  Instructional  System  Development; 
(b)  the  five-volume  Handbook  for  Designers  of  Instructional  Systems,  AFP  50-58;  (c)  the  re- 
cently published  TRADOC  PAMPHLET  350-30,  Interservice  Procedures  for  Instructional 
Systems  Development,  a comprehensive  description  of  approved  procedures  and  techniques 
to  be  followed  in  development  and  conduct  of  interservice  training;  and  finally  (d)  the  Grum- 
man report,  The  Development  of  a Systems  Approach  to  Training  and  Its  Application  to 
A-6E  TRAM Aircrew  Training.  1 

The  approach  Grumman  followed  in  the  Phase  I design  of  the  A-6E  TRAM  training 
program  was  in  close  accord  with  AFP  50-58.  It  consists  of  performance  of  the  first  five 
of  the  following  activities,  which  we  consider  as  adequately  representative  of  the  ISD  pro- 
cess. 

IDENTIFICATION  OF  JOB  PERFORMANCE  REQUIREMENTS.  The  first  step  in  the  process, 
and  a most  critical  one,  is  to  define  and  describe  the  tasks  that  must  be  performed  by  the 
A-6E  pilot  and  B/N  when  they  are  in  the  operational  environment.  The  tasks  must  be  com- 
prehensive and  stated  in  sufficient  detail  so  as  to  enable  the  identification  of  all  the  knowl- 
edges and  skills  that  are  involved  in  their  successful  completion.  Further,  it  is  also 
necessary  in  this  step  to  describe  the  operational  conditions,  within  which  these  jobs/tasks 
are  performed  in  order  to  properly  assess  their  impact  on,  or  interaction  with  performance. 
Finally,  appropriate  criteria  or  standards  which,  when  achieved,  represent  acceptable 
behavior  or  performance  of  the  job,  must  be  identified  and  agreed  upon.  (The  problem  of 
criteria  identification,  one  of  the  more  difficult  problems  encountered  in  applying  the  ISD 
process  is  discussed  more  fully  in  the  Rationale  portion  of  this  section).  Since  the 
training  program  development  depends  heavily  on  a good  job  performance  analysis,  the 
criticality  of  this  first  step  becomes  evident.  However,  because  it  is  an  iterative  process, 
some  omission  or  errors  can  be  corrected  downstream.  Job  performance  activities  and 
requirements  were  identified  through  a joint  team  effort  conducted  by  contractor  ISD 
training  specialists  and  VA-42  SMEs  (subject  matter  experts).  In  its  role  as  model  mana- 
ger for  the  program,  VA-42  assigned  combat  experienced  pilots  and  B/N's,  who  were  also 
veteran  instructors,  to  provide  technical  support,  monitoring  and  guidance  as  required 
throughout  the  various  activities  of  the  program.  The  division  of  work  betweem  the  Navy 
SMEs  and  the  Grumman  specialist  will  be  discussed  shortly. 

DETERMINING  TRAINING  REQUIREMENTS.  Not  all  the  tasks  that  collectively  define  the 
jobs  the  replacement  pilot  and  B/N  will  perform  in  the  operational  environment  need  to  be 
trained.  Some  of  the  simpler  motor  actions  may  already  be  in  their  repertoire.  Others 
may  already  have  been  trained  during  basic  flight  training.  Still  others  may  be  more 
appropriately  introduced  and/or  practiced  after  assignment  to  an  operational  squadron.  This 
second  major  activity  of  the  ISD  process  is  directed  toward  identifying  what  changes  in 
knowledges,  skills  and  attitudes  of  the  pilots  and  B/N's  entering  the  Readiness  Squadron 
are  required  to  enable  them  to  perform  their  identified  jobs.  This  activity  varies  in  magni- 
tude depending  on  the  variety  or  homogeneity  of  the  incoming  trainee  group  with  respect  to 
basic  training  or  entry  level  skills,  and  the  availability  of  this  data.  When  such  data  is  not 
readily  available,  screening  or  entry  level  testing  may  be  required. 

1 Campbell,  S. , Egan  J. , Morganlander,  M. , and  Schaefer,  R.  "Proposal  for  the  Develop- 
ment of  a Systems  Approach  to  Training  and  Its  Application  to  A-6E  (TRAM),  ....  Aircrew 
Training",  Grumman  Aerospace  Corp. , Bethpage,  N.Y.,  Feb.  1975. 
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A second  consideration  which  affects  the  size  of  the  effort  in  determining  tasks  to  be 
trained  is  the  extent  of  resources  available.  Reality  always  "intrudes"  to  place  limits  on 
how  many  resources  can  be  allocated  to  training.  These  constraints  might  for  example 
take  the  form  of  limited  dollars  and/or  equipment,  limited  instructor  availability,  limited 
time  to  conduct  training,  or  some  other  factors.  Rarely  is  there  sufficient  time  or  equip- 
ment to  train  all  the  tasks  that  should  be  trained  to  the  criterion  level  desired.  Depending 
upon  the  specific  constraints  concerned,  priorities  can  readily  be  placed  on  the  selection  of 
tasks  to  be  trained,  or  the  time  allotted  to  criterion  achievement.  Such  factors  as  time  crit- 
icality of  the  task,  its  frequency  of  occurrence,  and  the  likely  consequences  of  inadequate 
performances  can  be  used  to  set  up  a hierarchy  classification  for  tasks  to  be  trained. 

DETERMINE  CRITERION  OBJECTIVES.  The  term  criterion  objective  is  often  used  inter- 
changeably with  terminal  learning  objective,  and  is  meant  to  identify  the  learning  and  be- 
haviors which  are  demonstrated,  or  inferred  from  demonstrable  behavior,  upon  completion 
of  the  training  program.  The  first  task  of  this  phase  therefore  is  to  identify  all  those  be- 
haviors which,  when  performed  at  or  beyond  a pre-determined  minimum  criterion  level,  will 
indicate  that  the  training  program  has  achieved  its  goals.  These  behaviors  will  closely  re- 
semble, if  not  be  identical  to,  that  behavior  which  will  be  exhibited  in  the  operational  en- 
vironment. Its  demonstration  is  prima  facie  evidence  that  the  required  behavior  is  within 
the  individual’s  repertoire. 

In  addition  to  the  behavior  statement,  however,  a standard  or  criterion  or  performance 
for  that  terminal  behavior  must  also  be  established.  The  standard  generally  should  be  at 
a level  comparable  to  the  earlier  established  operational  standards.  These  latter  criteria 
however,  while  reflecting  an  acceptable  level  of  behavior  must  also  be  consistently  attain- 
able. In  other  words,  the  same  acceptable  level  of  performance  must  be  attained  for 
numerous  repetitions  of  a given  behavior.  In  the  training  situation,  to  achieve  a level  of 
confidence  that  the  desired  behavior  will  be  realized  operationally  some  feature  of  repeat- 
ability may  be  added  to  the  training  standard,  e.  g.  , ".  . . . in  6 of  7 trials  the  pilot  will 
. . . . , " or  ".  . . .theB/N,  after  the  third  flight,  will  perform  without  error  the  . ..." 

Alternatively,  a more  demanding,  severe  standard  might  be  required.  For  example, 
assume  a 10-foot  CEP  has  been  established  as  the  standard  of  acceptable  operational 
behavior.  Because  it  may  be  impractical  or  uneconomical  to  require  demonstration  of 
repeatability  of  performance,  e.g.  , 8 flights,  capability  for  acceptable  behavior  might  be 
inferred  if  the  trainee  achieved  a CEP  of  6 feet  or  less  on  4 successive  flights.  Extreme 
caution  should  be  exercised  before  this  type  of  standard  is  included.  A number  of  potential 
pitfalls  await  its  indiscriminate  use.  For  example,  system  delivery  errors  may  place  a 
severe  burden  on  crew  performance,  making  demands  far  in  excess  of  reasonable  perform- 
ance expectations.  Failure  to  achieve  criterion  level  performance,  or  inordinate  amounts 
of  practice  would  likely  result  from  this  situation.  Also,  it  might  be  entirely  impractical 
to  expect  operational  level  performance  from  Readiness  pilots.  This  level  might  not 
attained  until  considerable  additional  training  is  received  in  the  operational  squadron. 


A third  part  of  this  task  is  to  identify  the  conditions  under  which  the  job  perfor- 
mance will  be  exhibited  or  inferred.  These  conditions  can  be  of  two  kinds.  They  may 
represent  significant  aspects  of  the  operational  environment  which  should  be  simulated 
or  replicated  in  the  training  situation,  because  they  may  have  an  appreciable  effect  on 
performance.  Day  or  night  navigation  flights,  carrier  or  field  landings,  fully  operational 
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or  malfunctioning  aircraft  or  weapons  systems  are  examples  of  this  type  of  condition.  The 
second  kind  of  condition  refers  to  the  aids  or  prerequisite  data  available  to  the  crewman 
when  he  is  called  upon  to  demonstrate  criterion  level  behavior.  The  use  of  checklists,  radar 
scope  photography,  on-board  nomographs  and  the  like  represent  this  type  condition. 

The  terminal  behaviors  can  be  classified  into  various  types.  The  simplest  classifi- 
cation providing  a useful  subdivision  is  four  fold:  knowledges,  mental  skills,  physical 
skills  and  attitudes.  Once  typed,  the  behaviors  need  to  be  further  analyzed  for  prerequi- 
sites, that  is,  enabling  skills  and  knowledges  which  must  be  mastered  before  training  to 
ensure  that  terminal  objectives  can  be  completed.  Prerequisite  skills  and  knowledge  require- 
ments theoretically  should  continue  to  be  extracted  until  the  judgement  can  be  made  that 
all  materials  actually  required  in  the  learning  process  are  identified  and  further  extraction 
is  not  required,  either  because  as  the  trainee  already  possesses  the  knowledge  and  skill 
identified,  or  additional  material  would  be  superfluous  to  the  terminal  objective.  Obviously 
there  are  constraints  and  limitations  in  developing  the  training  program  just  as  there  are 
in  conducting  the  program.  A practical  limit  is  always  set  on  the  evolution  of  enabling 
objectives. 

The  identification  of  the  totality  of  learning  objectives  that  must  be  satisfied,  ter- 
minal and  enabling  alike,  provide  necessary  control  over  the  course  content  of  the  program. 
Unnecessary  or  irrelevant  instruction  both  in  on-going  programs  or  in  programs  under 
development  can  be  eliminated  or  avoided.  Their  very  specificity  further  enables  the 
trainee  to  have  a clear,  exact  understanding  of  what  is  and  will  be  expected  of  him.  All 
his  effort  can  be  properly  channeled  to  development  of  necessary  skills,  knowledges  and 
attitudes.  Spurious  information  attainment  or  extraneous  practice  can  be  avoided. 

DETERMINE  CRITERION  REFERENCE  TESTS.  This  activity  is  concerned  with  developing 
test  statements  which  will  be  direct  measures  of  the  criterion  level  behavior.  Tests  usually 
are  written  for  each  of  the  learning  objectives,  measuring  as  precisely  as  possible  the 
behavior  induced  as  an  outcome  of  the  instimction.  Tests  should  sample  behavior  at  the  same 
level  of  detail  as  presented  and  practiced  in  the  learning  situation.  Where  the  total  behavior 
cannot  be  adequately  demonstrated  in  test,  or  when  it  is  impractical  to  do  so,  the  tests 
should  at  least  include  sufficient  sampling  of  the  instructional  elements  and  behavior  so  as 
to  provide  confidence  in  the  adequate  mastery  of  all  the  material. 

On  occasion,  a test  constructed  to  measure  satisfaction  of  a terminal  learning  objec- 
tive can  also  simultaneously  satisfy  a number  of  associated  enabling  objectives.  In  these 
situations,  successful  test  demonstrations  imply  possession  of  the  prerequisite  knowledges 
and  skills. 

It  is  recognized  that,  when  relatively  complex  performance  measurement  is  involved, 
as  in  the  case  of  a pilot  and  B/N  flying  an  A-fiE  TRAM  aircraft  on  an  attack  mission, 
reliable  and  valid  measures  are  not  only  difficult  to  obtain,  but  are  often  relatively  low  in 
a statistical  sense.  Nonetheless,  they  are  the  key  to  making  the  necessary  decisions  about 
the  operational  readiness  of  the  student  as  well  as  the  assessment  of  the  quality  and  content 
of  the  instruction  itself. 

The  test  standards,  recognizing  the  build-up  of  skill  levels,  are  generally  less  de- 
manding early  in  the  training  sequence.  Final  performance  should  usually  reflect  the 
standards  required  in  the  operational  situation. 
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SELECTION  OF  INSTRUCTION  METHODS  AND  MEDIA.  This  activity  is  concerned  with  the 
organization  of  the  training  objectives  into  meaningful  instructional  units,  the  analysis  and 
selection  of  appropriate  training  media  to  support  the  instruction  and  the  initial  selection 
of  instructional  strategies  intended  for  implementation  in  the  units. 

Once  the  tasks  to  be  trained  have  been  identified  and  the  learning  objectives  specified, 
consideration  needs  to  be  given  to  decisions  about  how  the  tasks  ought  to  be  trained.  These 
include  the  grouping  of  materials  by  logical  relationships.  Also,  determining  what  infor- 
mation should  be  offered  as  prerequisite  to  other  materials.  Media  considerations  include 
how  best  to  use  media  to  provide  effective  and  efficient  training.  What  media  are  appro- 
priate for  what  instructional  units?  What  media  are  available  or  need  to  be  developed? 

The  organization  of  instruction  also  requires  a determination  as  to  what  kinds  of  instruc- 
tional settings  are  most  appropriate  for  the  conduct  of  training.  Should  the  training  be  self- 
taught,  self-paced,  or  individualized?  Should  it  be  conducted  in  a more  formal  setting  as 
classroom  or  trainer?  Should  it  emphasize  on  the  job  training,  i.e.  , with  the  aircraft  on  the 
line  or  in-flight?  Is  the  training  appropriate  for  inclusion  in  the  Readiness  Squadron 
curriculum  or  should  it  be  conducted  by  a support  organization  or  in  the  operational 
squadron  ? 

A number  of  factors  enter  into  the  making  of  appropriate  decisions.  These  include 
the  skill  levels  of  the  personnel  to  be  trained,  the  resources  and  facilities  available,  the 
number  of  personnel  entering  the  squadron,  the  number  of  training  activities  that  need  to 
be  integrated  with  other  Navy  units  outside  the  Readiness  facility,  e.  g.  , carrier  qualifica- 
tions, and  visual  weapons  deployment,  and  finally,  the  skill  with  which  appropriate  learning 
guidelines  and  principles  are  selectively  applied  to  the  instructional  unit  organization. 

A number  of  principles  proven  effective  across  ali  types  of  learning  situations  can  be 
applied  generally.  These  include,  for  example,  principles  of  feedback  application,  the  use 
of  practice,  prompting  and  cueing  and  hand-on  training.  Other  principles  are  more 
appropriate  for  one  kind  of  learning  behavior  than  another.  For  example,  learning  may 
involve  grasp  of  a concept,  in  which  case  a number  of  illustrative  examples  may  be  present- 
ed, but  when  once  understood,  further  practice  is  not  required.  Conversely,  if  a manipu- 
lative skill  is  to  be  achieved,  continued  practice  might  be  required  to  keep  the  skill  at  a 

desired  level.  1 

Final  selection  of  learning  guidelines,  principles  and  instructional  strategies  is 
accomplished  during  the  development  phase  of  the  program  when  each  of  the  lessons  of  the 
curriculum  and  the  instructional  materials  are  produced. 

Where  a training  program  with  associated  media  already  exists,  it  is  appropriate  to 
review  the  syllabus  and  curriculum  to  determine  how  much  of  it  satisfies  the  ISD  training 
objectives,  and  how  much  of  the  material  is  potentially  useful.  This  process  should,  how- 
ever, be  conducted  in  consort  with  the  instructor  staff,  to  assure  that  what  appears  to  be 
identical  in  name,  is  identical  in  content.  Too  often,  like-sounding  subjects,  lessons  or 


For  a more  detailed  listing  of  learning  principles  and  guidelines  applied  to  training 
see  TRADOC  Pamphlet  .350-30;  Phase  III  Development,  Interservice  Procedures  for 
Instructional  Systems  Development,  8/75.  Also,  Willis,  M.  P and  Peterson,  R.O. , 
Deriving  Training  Device  Implications  from  Learning  Theory  Principles,  American 
Institute  for  Research,  Pittsburgh,  Pa,  Report  NAVTRADEVCEN  784  -1,  2 and  3, 
July  1961. 
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other  instructional  materials  contain  information  or  data  which  would  be  considered  as 
superfluous  or  unjustified  by  the  ISD  process,  or  which  would  have  the  wrong  emphases. 
Where  the  curriculum  materials  do  satisfy  ISD  training  objectives,  some  measure  of 
instructional  material  validation  can  likely  be  obtained. 

At  the  same  time  the  curriculum  is  being  reviewed,  it  is  also  appropriate  to  survey 
or  otherwise  analyze  the  supporting  media  available  in  inventory.  While  final  media 
selection  awaits  results  of  instructional  materials  organization  and  sequencing,  as  well 
as  instructional  strategies  employed,  a preliminary  feel  as  to  what  is  available,  or  what 
might  need  to  be  purchased  commerically  is  valuable  to  the  program  developer.  By  sorting 
the  behavioral  objectives  in  terms  of  the  capability  for  satisfaction  by  generic  media  type, 
early  consideration  can  be  given  to  the  type  and  eventual  number  of  hardware  units  that 
are  likely  to  be  required.  In  addition,  important  instructional  features  within  the  learning 
objectives  are  often  identified. 

DEVELOPMENT  OF  INSTRUCTIONAL  MATERIALS.  This  activity  consists  of  the  actual 
production  and  integration  of  the  instructional  material  which  will  be  used  in  the  training 
program,  e.g.,  sound-slide  programs,  charts,  curricula.  It  and  all  subsequent  major 
ISD  activities  are  to  be  performed  during  Phase  II. 

VALIDATION  AND  REVISION  OF  THE  INSTRUCTIONAL  MATERIALS.  This  effort  has 
three  goals:  (1)  to  ensure  the  technical  adequacy  and  correctness  of  the  instructional 
materials  produced;  (2)  to  obtain  preliminary  concurrence  and  evidence  that  the  proposed 
training  is  appropriate  for  achieving  the  training  objectives;  (3)  to  verify  that  the  desired 
learning  is  indeed  the  result  of  training  with  the  instructional  materials. 

The  validation  process  usually  consists  of  a series  of  trials,  first  with  a few  subject 
matter  experts,  followed  by  small  group  and  large  group  trials.  The  first  trial  usually  is 
directed  toward  goal  1 while  the  latter  trials  are  intended  to  achieve  goals  2 and  3.  The 
subject  personnel  for  the  latter  trials  are  sample  group  representations  of  the  trainee 
population.  Often  pre-  and  post-tests  need  to  be  prepared  as  measures  of  performance 
improvement  resulting  from  training  with  the  instructional  materials.  Revisions  to  this 
material  can  be  made  after  each  of  the  three  trials,  and  should  be  completed  before  the 
production  run  on  the  instructional  matter. 

CONDUCT  OF  THE  TRAINING  PROGRAM.  This  task  consists  of  the  actual  scheduling  and 
implementation  of  the  training  program.  In  addition  to  the  presentation  of  the  curriculum 
materials,  it  will  include  instructor  preparation  and  training  in  the  use  of  instructional 
materials  and  the  daily  scheduling  of  personnel,  instructors,  facilities  and  training  re- 
sources. An  important  part  of  this  activity  is  the  identification  of  and  the  planning  for 
acquiring  of  field  data  to  evaluate  the  on-going  program. 

EVALUATION  OF  THE  INSTRUCTIONAL  PROGRAM.  Once  the  instructional  program  has 
been  completed  an  assessment  oi  how  well  that  program  prepared  the  trainee  group  to  per- 
form in  the  operational  situation  needs  to  be  made.  Detailed  performance  records  which  can 
be  compared  with  training  requirements  and  standards  need  to  be  collected.  These  records 
often  need  to  be  supplemented  by  surveys  made  in  the  operational  squadrons,  to  complete 
the  training  program  assessment  and  to  ensure  accurate  interpretation  of  the  performance 
records  obtained. 
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RATIONALE  FOR  METHODOLOGY  USED 

The  ISD  methodology  followed  in  designing  the  A-(>E  TRAM  aircrew  training  program 
was,  in  general,  that  described  in  the  preceding  section.  However,  there  were  a few 
problems  involved  and  decisions  made  about  the  approach  which  warrant  further  clarification. 

These  were: 

• The  role  of  the  SME  in  the  ISD  process 

• Media  recommendations  and  the  Lesson  Specification 

• Identification  and  use  of  performance  standards 

THE  ROLE  OF  THE  SME  IN  THE  ISD  PROCESS.  Considerable  thought  was  given  as  to  how 
one  might  best  use  the  time  and  expertise  of  the  SMEs  in  the  ISD  process.  Mindful  of  the 
fact  that  the  SME  is  at  the  same  time  one  of  the  instructors  who  will  eventually  be  con- 
ducting the  training,  the  impulsive  approach  might  attempt  to  involve  the  SME  as  much  as 
possible,  even  to  the  extent  of  making  him  an  ISD  "expert".  While  this  approach  has  the 
obvious  benefit  of  having  the  instructor  prepare  his  own  training  program,  it  suffers  from 
some  major  flaws. 

First,  it  places  a restriction  on  the  selection  of  SMEs.  A considerable  amount  of 
time  would  have  to  be  invested  in  initiating  the  SMEs  to  the  ISD  philosophy  and  in  training 
them  to  conduct  the  process.  This  investment  would  dictate  that  the  SME  continue  the  pro- 
cess to  completion.  Rotation  and  reassignment  of  SMEs  due  to  completion  of  tour  of  duty 
would  result  in  costly  retraining  to  bring  the  new  SMEs  "up  to  speed".  Secondly,  because 
both  instruction  in  the  ISD  process  and  production  of  the  training  program  via  this  approach  is 
a relatively  time  consuming  process,  the  SME  who  performs  the  ISD  becomes  almost 

totally  lost  to  the  squadron  for  other  duties  during  the  entire  time  required  for  the  program  ; 

design  phase  (Phase  I),  and  likely  much  of  the  time  during  the  development  phase  (Phase  II). 

Thirdly,  the  very  nature  of  the  ISD  process  tends  to  become  a real  problem,  highly 
attitudinal  in  nature,  for  the  SME.  Performing  the  ISD  requires  a detailed,  painstaking 
attention  to  behavioral  activities,  often  at  a most  elemental  level.  The  process  is  thorough, 
logical  and  complete,  but  tedious  and  laborious,  in  fact,  it  is  considered  by  some  as  oft- 
times  boring  in  its  day-bv-day  activity.  The  SME  on  the  other  hand  is  an  experienced, 
successful  Naval  Flying  Officer.  By  aptitude  as  well  as  by  interests,  if  not  almost  by 
definition,  he  is  an  intelligent,  dynamic  and  highly  motivated  individual,  who  greatly  prefers 
doing,  which  in  this  case  is  spelled  f-l-y-i-n-g,  to  sitting  at  a desk  preparing  behavioral 
objectives.  He  has  not  been  selected  and  trained  at  considerable  cost  to  the  government  to 
spend  his  time  as  an  educational  specialist,  at  possibly  considerable  cost  to  himself. 

At  the  opposite  end  of  the  spectrum  of  SME  involvement,  one  might  attempt  to 
minimize  any  interference  with  the  SME  squadron  duties,  limiting  his  participation  to 
consultation  on  technical  matters,  and  review  of  program  design  material  developed  by  the 
contractor  training  specialists.  The  principal  danger  of  this  approach  is  that  the  task 
analysis  and  resulting  training  materials  will  lack  the  operational  flavor  and  subtle  essentials 
that  can  only  be  recognized  and  appreciated  by  the  professional.  Just  as  it  seenis  inap- 
propriate to  require  the  SME  to  be  an  ISD  specialist,  so  too,  it  is  equally  inappropriate 
for  the  training  specialist  to  attempt  to  achieve  a level  of  technical  competence  sufficient  to 
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perform  a satisfactory  analysis  of  the  operational  activities  of  the  flight  crew.  Because 
the  ISD  success  or  failure  is  heavily  dependent  upon  the  validity  and  completeness  of  the 
first  major  step,  the  job  analysis,  the  SME  must  be  sufficiently  involved  to  ensure  that  the 
training  specialist  has  an  adequate  understanding  of  these  activities. 

A secondary  danger,  likely  to  be  encountered  if  the  training  expert  conducts  ISD  in 
isolation  from  the  SME  and  other  squadron  personnel,  is  that  the  failure  of  the  contractor 
training  specialist  to  communicate  adequately  to  these  key  personnel  the  intent  of  the  ISD 
process,  could  result  in  a suspicion  of  the  validity  and  merit  of  the  instructional  material. 


Grumman  opted  to  recommend  SME  participation  somewhat  between  these  two 
approaches.  Heavy  participation  was  requested  of  VA-42  SME  personnel  during  the  job 
analysis.  Grumman  ISD  specialists  and  Navy  SMEs  worked  together  as  a team  alternately 
on-site  at  VA-42  and  at  the  contractor's  facilities  until  comprehensive,  although  preliminary 
mission  oriented  nominal,  malfunction  and  emergency  pilot  and  B/N  task  descriptions  were 
identified.  At  that  point  participation  from  VA-128  personnel  was  enlisted.  The  VA-128 
SMEs  provided  a validation  of  the  preliminary  task  listings.  Only  minor  modifications  were 
required  to  achieve  mutual  squadron  agreement. 


Because  the  SME  role  was  that  of  the  technical/ operational  expert,  two  changes  in 
SME  assignments  were  accomplished  without  perturbing  the  process.  The  first  occurred 
in  VA-42  tour  of  duty  reassignments  of  both  the  pilot  and  B/N  SMEs  during  task  listing 
development.  The  transitional  phase  was  accomplished  smoothly.  The  second  change 
was  in  assignment  of  pilot  support  from  VA-128.  Short-term  assignment  during  the  valida- 
tion of  the  task  listings  and  during  the  identification  of  performance  standards  and  opera- 
tional conditions  did  not  cause  undue  delay  as  SMEs  were  not  called  upon  to  perform  in 
areas  outside  their  expertise. 

SMEs  also  provided  liaison  in  surveying  media  available  within  their  respective 
squadrons  and  again  worked  briefly  as  a team  during  initial  SBO  development.  They 
participated  in  the  preparation  only  to  the  extent  necessary  to  give  them  an  appreciation 
of  that  part  of  the  process.  Consequently,  they  were  able  to  judge  the  correctness  of  the 
contractor's  interpretation  of  the  technical  subject  matter  proposed  for  training. 

They  also  provided  insightful  recommendations  about  enabling  skills  and  knowledges 
that  needed  to  be  trained  to  achieve  the  behavioral  objectives. 

The  last  major  interface  of  the  Navy/Grumman  ISD  team  occurred  in  the  preparation 
of  the  Lesson  Specifications.  Contractor  training  experts  presented  a "straw-man”  syllabus 
proposing  how  the  various  SBOs  and  lessons  might  be  combined  into  a training  program. 

In  addition  to  suggesting  recommendations  for  the  sequencing  of  lessons,  the  SMEs  pro- 
vided consultation  on  existing  curricula,  and  such  information  as  phases  of  training,  an- 
cillary training  which  replacement  pilots  and  B/Ns  receive  during  their  time  in  the  Readiness 
Squadron,  and  coordinated  training  phases  such  as  the  Visual  Weapons  (VISWEPS)  phase. 

With  this  information  the  development  of  Lesson  Specifications  could  then  be  completed. 

This  use  of  the  SMEs  proved  to  work  quite  satisfactorily.  Appropriate  and  sufficient 
support  was  provided  to  contractor  personnel,  while  interruptions  to  the  daily  routines  of  the 
squadrons  appeared  to  be  small  but  acceptable.  The  total  time  required  of  all  SMEs  during 
Phase  I is  indicated  in  .Section  IV. 
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MEDIA  RECOMMENDATIONS  AND  LESSON  SPECIFICATIONS.  The  recommendations  re- 
sulting from  the  media  analysis  reflect  contractor  judgement  as  to  what  might  reasonably  be 
expected  with  regard  to  media  availability  and  use.  It  was  thought  to  be  totally  impractical 
to  merely  recommend  the  best  type  or  mix  of  media  to  satisfy  particular  learning  objec- 
tives without  regard  to  the  extent  of  new  media  development  that  might  be  required.  On 
the  other  hand,  to  limit  recommendations 'to  available  media  could  seriously  curtail  the 
quality  of  training  that  might  be  achieved. 

The  solution  to  this  problem  was  as  follows.  The  recommendations  made  in  the  media 
analysis2,  represent  what  is  considered  to  be  the  preferred  type  or  mix  of  media  in  satisfy- 
ing the  various  learning  objectives.  In  instances  where  these  represent  media  which  are 
not  currently  available,  for  example  WST  with  a land-mass  visual  capability,  or  which, 
although  available,  lack  a particular  capability,  for  example,  SCEPTR,  a procedures 
trainer  for  B/N  Weapons  System  Training,  alternatives  are  listed.  These  alternatives 
are  acceptable  although  they  may  provide  a less  desirable  training  mode  or  a more  costly 
mode.  For  example,  a WST  with  visual  capability  could  provide  FLR  visual  and  display 
imagery  and  carrier  landing  training  capability,  which  otherwise  would  have  to  be  performed 
on  the  operational  aircraft.  Functional  descriptions  of  these  trainers,  preliminary  in 
nature,  are  included  in  the  media  report. 

In  other  instances,  media  recommendations  assume  that  existing  media  will  be 
partially  or  substantially  updated.  Here  it  is  anticipated  that  such  updating  will  be  accom- 
plished as  part  of  Phase  II.  This  category  includes  sound-slide  program  updating  and 
updating  of  the  pilot  SCEPTR  to  an  A-6E  TRAM  configuration.  It  is  further  expected  that 
some  new  media  will  be  developed  during  Phase  II.  Included  in  this  category  are  sound- 
slide  programs,  charts,  video  tapes  and  similar  "software". 

This  approach,  matching  SBOs  and  media  was  carried  through  to  Lesson  Specifica- 
tions. Here,  however,  it  was  felt  paramount  to  present  media  recommendations  which 
would  allow  a training  program  to  be  developed  and  conducted.  Insisting  on  media  which  may 
or  may  not  be  developed  in  the  near  future  does  not  bring  us  to  that  goal.  Therefore  all 
media  recommendations  shown  in  the  Lesson  Specifications  are  limited  to  those  which  can 
be  prepared  for  an  imminent  training  program.  The  potential  benefits  to  the  training  pro- 
gram identified  through  the  ISD  process  will  be  largely  wasted  however  unless  the  media 
recommendations,  to  wit,  visual  capability  added  to  WST,  and  development  of  pilot  and 
B/N  procedures  trainers,  are  implemented. 

THE  IDENTIFICATION  AND  USE  OF  PERFORMANCE  STANDARDS.  A key  feature  of  the 
ISD  process  is  the  emphasis  placed  on  clear,  well-defined  behavioral  objectives.  The  trainee 
in  an  ISD  based  training  program  should  understand  as  precisely  as  possible  not  only  what  is 
expected  of  him  and  what  he  must  learn  and  do  to  get  there,  but  also  what  criterion  level  of 
performance  he  must  exhibit  to  demonstrate  he  has  achieved  that  expectation.  Where  tasks 
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Requirements  (Media  Analysis),  A-6E  (TRAM)  Aircraft,  Grumman  Aerospace  Corp. , 
Bethpage,  N.  Y.,  Report  ISD  (SAT)  0004,  Feb.  1976. 

G.  Graham,  and  J.  Feddern,  Lesson  Specification  Documents,  A-6E  (TRAM)  Aircraft, 
Grumman  Aerospace  Corp.,  Bethpage,  N.  Y. , Report  ISD  (SAT)  0006,  April  1976. 
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or  activities  are  relatively  simple,  or  where  they  lend  themselves  to  readily  quantifiable 
or  measurable  standards,  little  problem  is  experienced.  Typical  classroom  learning 
activities  permit  reasonably  well-defined  performance  standards.  For  example,  the  number 
of  items  correct  minus  number  of  items  incorrect  in  a written  test  yields  a score  which 
can  lie  easily  compared  with  a pre-set  standard. 

Few  of  the  pilot  and  B/N  activities  lend  themselves  to  such  a simplistic  scoring 
scheme.  In  general  the  activities  of  the  aircrewman  involve  a sequence  of  coordinated, 
complex  and  integrated  activities  directed  towards  a two-faceted  goal:  performing  the 
assigned  mission  successfully  and  safely.  Within  the  boundaries  of  this  practical  goal  much 
individual  variation  is  acceptable.  The  instructor  very  often  is  called  upon  to  "rate"  the 
pilot  and  B/N  on  rather  large-scale  categories  such  as  instrument  procedures,  formation 
procedures,  fuel  planning  and  monitoring,  landings,  headwork  and  agressiveness.  These 
ratings  are  made  on  a series  of  flights  wihin  a given  phase  of  training.  The  ratings  are 
qualitative  in  nature  with  the  replacement  pilot  or  B'N  performance  being  located  on  a scale, 
ranging  from  above  average  through  marginal  or  unsatisfactory.  Documented  information 
on  specifically  what  should  be  demonstrated  by  the  replacement  pilot  to  achieve  a "SAT" 
rating  is  not  generally  available. 


And  yet,  surprisingly,  or  perhaps  not  so  surprisingly,  there  is  a realtively  good 
correlation  among  instructors  in  rating  trainee  performance.  Experience  has  shown  that 
given  a set  of  veteran  instructors,  all  evaluating  the  same  performance,  markedly  good 
agreement  would  likely  be  found.  Further,  without  having  verbalized  the  criterion  per- 
formance desired,  instructors  do  tend  to  transmit  to  the  "rookie"  their  expectations,  e.g.  , 
a turn  point  not  precisely  navigated,  a coordinated  turn  too  loose,  an  approach  too  high. 

The  performance  standard  demanded  is  zero  error,  whether  flying  the  ball,  navigating 
to  a check  point  or  placing  cursors  squarely  on  target.  Some  composite  deviation  from 
zero  error,  if  small  enough  in  the  judgment  of  the  skilled  instructor  represents  satisfactory 
performance.  The  instructor  continually  attempts  to  narrow  down  the  corridor  of  deviant 
performance,  to  reduce  the  range  of  error,  until  the  performance  is  judged  at  least  average- 
a zeroing  in  on  acceptable  performance. 

The  lack  of  objective  performance  standards  became  a problem  during  the  process  of 
developing  the  SBOs.  The  ISD  process  called  for  the  identification  of  quantitative  perform- 
ance standards,  both  for  operational  performance  as  well  as  for  criterion  test  behavior. 
Although  the  SMEs  were  extremely  cooperative  and  helpful,  not  very  many  documented 
standards  were  available.  There  was  generally  concurrence  between  Navy  and  contractor 
team  members  that  the  more  precisely  and  clearly  the  standards  could  be  defined  for  the 
trainee,  the  more  appropriate  would  be  his  attempts  at  achieving  that  behavior.  There  was 
somewhat  less  concurrence  on  the  likelihood  of  developing  such  standards  or  their  suit- 
abilitv  for  use  with  the  kind  of  activities  involved.  Further,  a natural  concern  existed 
that  should  operational  standards  Ije  identified,  they  would  promptly  be  considered  as  "cast 
in  concrete"  before  their  assessment,  and  that  they  would  either  be  prematurely  intro- 
duced into,  or  rejected  by,  training  squadrons;  or  that  they  might  be  misused  or  adapted 
incorrectly  in  the  training  program. 


,o 
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After  careful  examination  of  the  alternatives  the  decision  was  reached  to  see  what 
success  might  be  achieved  in  developing  these  quantitative  standards.  Attempts  at  describ- 
ing performance  levels  soon  indicated  that  the  instructors  did  indeed  have  an  internalized 
criterion  set  that  they  used  in  rating  their  students.  Open  discussion  of  these  eventually 
yielded  a set  of  standards  that  were  then  associated  with  the  SBOs  and  used  in  developing 
the  Lesson  Specifications.  There  was  a general  tendency  to  want  to  "open  up"  the  standards 
somewhat,  but  those  identified  have  general  squadron  SME  concurrence  as  a "preliminary 
set".  They  are  found  mainly  in  the  area  of  B/N  activities,  which  in  general  tended  to 
allow  examination  of  more  discrete  segments  of  activities  than  did  the  pilot  tasks. 
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SECTION  III 
IMPLEMENTATION 


TASK  ANALYSIS 

INTRODUCTION.  The  job/task  analysis  phase  of  the  ISD  process  was  performed  by  a team 
consisting  of  Navy  Subject  Matter  Experts  (SMEs),  and  contractor  training  psychologists, 
educational  specialists  and  flight  test  personnel.  The  goal  of  this  phase  was  to  identify  the 
tasks  that  an  A-6E  TRAM  pilot  and  B/N  would  be  called  upon  to  perform  in  the  operational 
environment.  These  tasks  were  to  be  described  in  sufficient  depth  to  permit  an  identifica- 
tion of  the  underlying  skills  and  knowledges  which  the  crewmen  must  possess  in  order  to 
successfully  perform  the  tasks. 

The  job/ task  analysis  was  preceded  by  a compilation  and  review'  of  engineering,  flight 
test  and  operational  documentation  of  the  A-6E.  It  consisted  of  in-house  A-6  data  obtained 
from  various  contractor  departments  and  Navy  documentation  obtained  from  VA-42.  The 
compilation  formed  a library  encompassing  the  full  range  of  aircraft  and  on-board  equip- 
ment information,  operation  and  functional  descriptions.  This  library,  enhanced  by  con- 
tinuing dialog  with  Navy  SMEs  represented  the  technical  source  reference  materials  for 
this  and  subsequent  phases  of  the  study.  This  data  was  later  supplemented  by  documents 
from  VA-128. 

To  further  ensure  consistency  with  the  design  goal  of  creating  an  aircrew  training 
program  specific  to  the  performance  requirements  of  an  operational  environment,  a mis- 
sion scenario  was  developed  to  aid  the  generation  of  job/task  descriptions  and  subsequent 
analysis.  The  scenario  consisted  of  a typical  sequence  of  major  events  that  would  take 
place  during  a nominal  A-6E  mission.  Generally,  it  began  with  mission  briefing  and 
planning,  followed  by  aircraft  preparation  for  flight,  take-off  and  climb  operations,  a tac- 
tical phase,  aircraft  return,  descent,  landing,  post-flight  aircraft  check  and  shutdown  and 
debriefings.  Where  alternative  events  could  occur,  e.g.,  take-off  from  a field  vice  a 
carrier,  a parallel  event  flow  was  prepared.  The  tactical  phase,  which  involved  many  alter- 
natives depending  on  specific  mission  and  weapon  delivery  options,  was  expanded  to  provide 
a logic  flow  approach,  specifying  the  conditions  leading  to  the  choice  and  exercise  of  various 
options.  Included  in  the  tactical  portion  of  the  scenario  was  air  combat  maneuvering  and 
flight  to  and  from  target  area. 

This  scenario  was  then  used  in  generating  the  nominal  pilot  and  B/N  activities  per- 
formed in  accomplishing  the  events.  Finally,  since  non-nominal,  malfunction  and  emerg- 
ency situations  might  occur  at  various  times  in  a mission,  these  were  identified  independent 
of  the  mission  scenario  and  the  pilot  and  B/N  activities  dealing  with  these  situations  were 
identified  and  described.  It  was  recognized  that  certain  aspects  of  malfunctions  were  mis- 
sion dependent,  i.e.  , had  different  corrective  action  or  aircrew  responses  according  to 
where  in  the  mission  they  occurred,  for  example,  engine  failure  prior  to  take-off,  vs  engine 
failure  in  flight.  In  such  instances  pilot  actions  under  both  conditions  were  identified. 

Applying  the  ISD  principle  of  employing  a methodological  and  systematic  process  in 
the  identification  of  job/task  elements,  the  task  analysis  effort  began  by  establishing  a 
hierarchy  for  describing  the  pilot  and  B/N  behaviors  during  a typical  mission.  The 
hierarchy,  as  structured,  generally  included  three  levels  of  description.  It  was  generated  by 
defining  each  phase  of  flight  in  terms  of  major  mission  events,  the  tasks  which  comprise  the 
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events,  and  finally,  the  steps  which  describe  the  incremental  actions  an  aircrewman  must 
take  to  complete  a task.  Occasionally  step-level  activities  required  an  additional  subdivi- 
sion. This  necessity  arose  where  the  complexity  of  the  crewman's  activity  was  such  that 
further  description  was  needed. 

The  identification  and  description  of  the  crew  activities  was  accomplished  in  a series 
of  working  sessions  with  the  SMEs.  Following  each  session  the  training  and  educational 
specialists  regrouped  to  summarize,  organize  and  arrange  the  information  discussed  into 
a preliminary  task  listing.  This  listing  was  reviewed,  modified  and  updated  via  SME  reviews 
until  a technically  satisfactory  listing  was  completed.  This  listing  was  then  reviewed  by 
SMEs  from  VA-128. 

Following  the  development  of  task  listings,  an  analysis  of  the  training  requirements  re- 
lated to  the  tasks  was  conducted  via  a series  of  interviews  with  the  SMEs  from  both 
squadrons.  A form  entitled  the  Task  Analysis  Record  Sheet  (TAR)  had  previously  been  pre- 
pared to  aid  in  collecting  this  data.  As  had  been  anticipated,  the  detailed  analysis  resulted 
in  the  generation  of  thousands  of  task  statements  and  a substantial  amount  of  related  data. 
Recognizing  that  manual  manipulation  of  such  huge  amounts  of  data  would  be  exceedingly 
laborious  and  time  consuming,  an  electronic  data  processing  system  was  established,  the 
commercially  available  MARK  IV  General  Data  Management  System.  Development  of  this 
system  allowed  the  sorting  and  assembling  of  both  task  descriptions  and  supporting  data. 

THE  PROCESS.  Essential  to  the  analysis  process  was  the  establishment  of  a selective  yet 
comprehensive  library  of  source  data.  Acquisition  of  documentation,  an  activity  which 
continued  for  the  duration  of  the  study,  began  within  the  contractor  organization.  Engineer- 
ing, Life  Sciences,  Publications  and  Flight  Test  Department  files  were  reviewed  for  infor- 
mation pertaining  to  A-6  and  A-6E/TRAM  A/C.  The  result  was  a complete  compilation  of 
A- 6 NATOPS  manuals,  Grumman  Data  Packages  for  the  TRAM  configuration,  specifica- 
tions for  the  A-6E  Weapons  System  Trainer,  and  other  related  engineering  analysis  and 
design  data.  A survey  of  these  documents  provided  contractor  personnel  with  an  understand- 
ing of  the  A-6  Pilot  and  B/N  operational  activities  sufficient  to  develop  our  initial  thoughts 
as  to  how  the  Task  Analysis  was  to  be  accomplished. 

It  was  obvious  that  the  Navy/contractor  ISD  team  which  was  to  be  formed  to  perform 
the  analysis  was  to  be  composed  of  personnel  with  diverse  professional  backgrounds.  The 
contractor  personnel  assigned  to  the  ISD  team  had  the  greater  proportion  of  their  work 
experience  in  the  field  of  education  and  training,  some  experience  in  aircraft  design  but  a 
relatively  limited  understanding  of  (and  no  experience  in)  the  operation  of  the  A-6  aircraft. 

On  the  other  hand  the  SMEs  had  extensive  flight  experience  with  limited  knowledge  of  the 
ISD  process.  It  was  recognized  that  any  planned  approach  for  building  an  integrated  work 
team  required  a blending  of  these  diverse  talents  through  the  establishment  of  a common 
frame  of  reference. 

For  the  VA-42/contractor  team  this  frame  of  reference,  was  a jointly  developed  typi- 
cal mission  scenario.  Ancillary  to  the  development  of  the  scenario,  but  just  as  important, 
was  the  education  of  SMEs  in  the  ISD  process.  With  this  understanding  in  mind  the  work 
activity  was  planned  according  to  the  following  procedure: 

a.  Set  forth  an  approach  for  development  of  a typical  mission  scenario. 

b.  Conduct  informal  briefings  of  SMEs  on  the  ISD  philosophy  and  process. 
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c.  Collaborate  with  SMEs  in  the  development  of  the  mission  scenario. 

d.  Develop  a job/task  hierarchy  as  a result  of  working  sessions  with  naval  aircrew- 
men  with  A-6  operational  experience. 

e.  Conduct  a job/ task  analysis  resulting  in  a task  listing. 

f.  Develop  a computer-based  sort  system  to  aid  in  the  manipulation  cf  job/task  data. 

g.  Initiate  ongoing  technical  monitoring  of  contractor  activities  by  SMEs. 

Initially,  the  contractor  ISD  specialists  developed  a typical  mission  flow  chart  using 
as  principal  technical  references: 

• A-6  NAVAIR  01-85-ADA-1B  NATOPS  Pocket  Check  List 

• A-6  NAVAIR  01-85-ADA-l  NATOPS  Flight  Manual 

• Grumman  Data  Packages  for  Integrated  Weapons  System  Theory  - Preproduction 
TRAM  A/C 

• Grumman  Flight  Test  Personnel 

The  flow  chart  provided  a pictorial  representation  of  a typical  mission  consisting  of 
nominal  mission  activities  (those  activities  common  to  all  missions)  and  tactical  activities 
(those  activities  which  are  unique  within  a mission  as  required  by  weapons  mix  and/or  de- 
livery mode).  The  first  evolution  of  the  flow  chart  identified  major  pilot  and  B/N  activities 
such  as  Preflight  Checks,  Exterior  Inspection,  Prestart  Procedures,  etc.  This  level  of 
aircrew  activity  identification  was  designated,  for  the  task  hierarchy,  as  EVENT  level 
activities.  The  flow  was  completed  from  preflight  through  aircraft  land  or  carrier  re- 
covery and  post  flight  operation.  1 Nominal  flight  activities  were  comparatively  well- 
defined  for  the  EVENT  level  in  that  there  was  sufficient  information  available  from 
Grumman  Flight  Test  personnel  and  the  reference  documents.  However,  the  tactical 
activities  were  superficially  identified;  it  was  apparent  that  a substantial  SME  input  would 
be  required  for  this  portion  of  the  job/task  definition. 

A second  iteration  of  the  flow  was  then  accomplished  to  further  expand  each  of  the 
event  level  activities  in  terms  of  crew  TASKS  which  comprise  the  EVENT  and  the  incre- 
mental STEPS  taken  to  accomplish  each  TASK. 

These  early  ISD  activities  served  a threefold  purpose:  first,  to  educate  contractor 
educational  specialists  with  regard  to  pilot  and  B/N  operational  duties;  secondly,  to  estab- 
lish a technical  frame  of  reference  which  was  mutually  understood  by  both  contractor  person- 
nel and  the  Navy  subject  matter  experts;  and  thirdly,  to  establish  a method  by  which  the 
identification  of  job/ task  data  would  be  pursued. 

Having  developed  the  mission  scenario  to  a point  where  there  was  confidence  in  both 
the  validity  of  the  approach  and  the  information  contained  therein,  V'A-42  subject  matter  ex- 
perts were  consulted  for  a technical  review  of  the  work  accomplished  and  to  discuss  their 
role  and  technical  input  for  further  definition  of  operational  activities. 


1 The  scenario  was  developed  as  a working  tool,  and  as  such  is  unpublished. 
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The  objectives  of  this  first  technical  conference  with  VA-42  subject  matter  experts 
include  the  following: 

• Technical  verification  and  update  of  work  accomplished 

• Identification  and  definition  of  additional  nominal  mission  events 

• Identification  and  definition  of  activities  within  each  event 

• Definition  of  the  independent  and  interdependent  aspects  of  the  job/tasks  identified 

• Acquisition  of  additional  documentation  specifically  related  to  aircraft  operation 

The  conference  was  conducted  at  VA-42  NAS  Oceana,  Virginia.  Two  contractor  training 
specialists  and  two  Navy  subject  matter  experts  (VA-42  pilot  and  B/N)  were  present.  The 
mission  scenario  was  reviewed  with  the  SMEs  for  format  and  content  with  all  four  team 
members  participating.  This  total  team  effort  continued  through  the  identification  of  all 
major  nominal  events.  The  team  then  split  into  two  subgroups,  one  to  work  principally  on 
pilot  activities,  the  other  B/N  activities.  Each  subgroup  consisted  of  one  subject  matter 
expert  and  one  training  specialist,  with  the  SMEs  talking  the  latter  personnel  through  each 
event  using  their  personal  knowledge  and  Navy  documentation  as  references,  and  assuring 
themselves  that  the  training  specialists  had  a sufficient  understanding  of  the  operations 
involved.  Periodically  the  subgroups  reconvened  as  a team  for  status  assessments  which 
also  served  as  reviews  and  technical  cross  checks  of  work  accomplished,  and  for  discus- 
sions regarding  interdependent  crew  activities.  The  educational  specialists  carefully 
documented  these  working  group  interviews  for  later  use  during  the  writing  of  the  task  and 
step  levels  of  the  job/task  hierarchy. 

The  results  of  the  conference  were:  (1)  the  mission  scenario  was  completed  for  all 
major  nominal  events;  (2)  the  SMEs  provided  an  elaboration  of  the  activities  occurring 
within  each  event  including  identification  of  independent  and  interdependent  tasks; 

(3)  initial  discussions  regarding  the  further  development  of  tactical  events  were  held;  and 

(4)  the  SMEs  provided  additional  reference  materials,  i.e. , VA-42  A-6  Training  Syllabus, 
briefing  guides,  lecture  outlines,  student  progress  folders,  and  samples  of  various  forms 
and  charts  used  within  the  squadron,  e.  g. , aircraft  inspection  form  (yellow  sheets)  and 
navigation  charts. 

Using  the  newly  acquired  information,  contractor  ISD  specialists  completed  the  job/task 
hierarchy  for  the  nominal  mission  activities.  Eigure  2,  a representative  sample  of  Event 
25,  Pre-Approach,  is  included  as  an  example  of  how  the  information  obtained  from  the 
SMEs  was  organized  and  recorded  by  the  training  specialists. 

Subsequent  conferences  followed  the  same  work  schedule,  i.  e.  , th  review  and  update 
of  contractor  accomplishments  followed  by  subgroup  and  total  team  activities  for  the  genera- 
tion of  new  material.  This  proved  to  be  a highly  efficient  and  productive  method  for  working 
with  the  SMEs. 

The  following  two  conferences  with  the  VA-42  SMEs  and  the  intervening  contractor  work- 
alone  periods  resulted  in  completion  of  the  typical  mission  scenario;  the  derived  prelimi- 
nary task  hierarchy  and  nominal  task  listing;  and  a preliminary  task  listing  of  emergency 
procedures  and  system  malfunctions.  About  the  time  of  the  second  conference  a transition 
of  SMEs  of  VA-42  was  begun;  it  w'as  completed  without  any  delay  to  the  work  effort  by  the 
time  of  the  third  meeting. 
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PRE-APPROACH 


E25 


PRE-APPROACH 

T1  CONTACT  APPROACH  CONTROL  AGENCY  (B/Il) 

51  SET  UHF  FREQUENCY  FOR  APPROACH  CONTROL 

52  IDENTIFY  A/C,  LOCATION  AND  ALTITUDE 

53  REQUEST  PERMISSION  FOR  AN  APPROACH 

Sh  INDICATE  LANDING  INTENTIONS 

55  RECEIVE  APPROACH  CLEARANCE  AND  EXPECTED  APPROACH 
CLEARANCE  TIME  (EAC) 

56  RECEIVE  HOLDING  INSTRUCTIONS  (IF  REQUIRED) 

T2  BEGIN  PENETRATION  (DESCENT)  CHECKLIST  (COORD) 

**NOTE:  BEFORE  DESCENT  AND  AFTER  PROLONGED  COLD** 

**SOAKING  AT  HIGH  ALTITUDE  COCKPIT  TEMPERATURE  ** 

**SHOULD  BE  INCREASED  AND  DEFOGGING  ACTIVATED  ** 


SI 

ADJUST  *AUTO  TEMP  CONT* 

POT 

..INCREASE  ( *HOT* ) 

(p) 

S2 

ADJUST  *DEFOG : AIRFLOW* 

POT 

. *ON* 

AND  INCREASE 
AS  REQUIRED 

(p) 

S3 

REFILE  ALTERNATE  (DIVERT  AIRFIELD) 
REQUEST 

CLEARANCE 
AS  REQUIRED 

(P  & B/N) 

SU 

CHECK  FUEL 

(B/N) 

(a)  VERIFY  MAXIMUM  GROSS  WEIGHT 

(b)  VERIFY  *FUEL  QUANTITY* 

IND.  . . 

.APPROPRIATE 
FOR  LANDING 

(c)  DUMP  FUEL  (IF  REQUIRED) 

(SEE 

E-» 

LTN 

C\J 

W 

(COORD) 

S5 

CHECK  WEATHER  (VIA  RADIO) 

(B/N) 

S6  CHECK  AND  SET  PRESSURE  ALTIMETER  FOR  FIELD  (P) 

PRESSURE  (RECEIVE  VIA  RADIO  FROM  APPROACH 
CONTROL)  AND  READ  BACK 


**CAUTION : UNLESS  CABIN  IS  DEPRESSURIZED,  DO** 

**NOT  ACTUATE  LANDING  GEAR  ABOVE  8000  FEET  ** 

Figure  2.  Sample  of  Task  Listing  for  Pre-Approach 
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At  this  point,  the  existing  ISD  team  took  time  out  to  welcome  aboard  a second  group  of 
SMEs,  a Pilot  and  B/N  from  VA-128  Readiness  Squadron  stationed  at  NAS  Widbev  Island, 
Washington.  Presentations  similar  to  those  provided  earlier  to  VA-42  S.MEs  were  given  the 
VA-128  members  - on  the  generic  ISD  process  and  the  application  of  its  methodology  to  the 
implementation  of  the  contracted  A-6E  TRAM  Phase  I study.  The  remaining  conference 
time  was  devoted  to  a detailed  review  of  the  task  listings.  Comments  and  specific  sug- 
gestions made  by  VA-128  SMEs  were  discussed  and  appropriate  modification  were  made. 
This  third  party  technical  review  was  in  effect,  a validation  of  the  w»ork  accomplished.  The 
addition  of  a second  group  of  SMEs  to  the  ISD  team  was  without  notable  problem  and 
proved  to  be,  over  the  course  of  the  study,  most  beneficial. 

The  task  listings  resulting  from  the  job  analysis  comprised  over  400  nominal  pilot 
tasks  each  with  an  average  of  approximately  10  steps;  70  airframe  emergency  sequences 
involving  an  average  of  7-10  steps  in  each,  35  system  malfunctions,  and  more  than  200 
nominal  B/N  tasks,  also  averaging  about  10  steps  each.  The  approved  task  listings  are 
contained  in  the  following  publications: 

• ISD  (SAT)  0003-1  Task  Listing,  A-6E  (TRAM)  Aircraft  Pilot  Procedures, 

Vol.  I,  Sept.  1975 

• ISD  (SAT)  0003-2  Task  Listing,  A-6E  (TRAM)  Aircraft  B/N  Procedures, 

Vol.  II,  (Confidential),  Sept.  1975 

• ISD  (SA1)  0003-3A  Task  Listing,  A-6E  (TRAM)  Aircraft  Airframe  Emergencies 
and  System  Malfunctions,  Volume  III,  Oct,  1975 

Classified  pilot  aircombat  maneuvering  tasks  were  included  in  Volume  II,  (already 
classified  confidential)  to  avoid  producing  a second  classified  document. 

The  listings  represent  tasks  for  which  training  needed  to  be  conducted  at  the  Readi- 
ness Squadron  level,  either  as  introductory  training  or  as  follow  on  to  training  received  at 
the  Air  Training  Command  for  the  pilots  or  in  previous  assignment  in  the  case  of  the  B/Ns. 

No  attempt  was  made  to  assess  the  appropriateness  of  the  Readiness  Squadron  loca- 
tion for  this  training,  or  to  judge  the  extent  of  entry  level  skills  and  knowledges  of  the  RP 
and  RBNs.  The  statement  of  che  SMEs  as  to  where  the  training  for  the  listed  tasks  occurred 
was  merely  accepted  without  questioning.  To  extend  the  analysis  either  to  earlier  assign- 
ment or  to  examine  operational  squadron  training  requirements  and  jurisdiction,  although 
highly  desirable,  was  beyond  the  bounds  of  the  current  effort. 

TASK  SELECTION 

Having  stabilized  the  task  listing  to  the  mutual  satisfaction  of  all  team  members  and 
having  gained  the  approval  of  the  NA VTRAEQUIPCEN  project  director,  the  next  step  in  the 
process  was  to  conduct  a training  requirements  analysis,  the  purpose  of  which  was  to 
examine  the  task  listing  and  to  identify  and  extract  those  factors  or  characterisitics  of  the 
task  which  influenced  the  training  which  would  be  conducted.  These  factors  include: 
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a.  The  aircrewman  responsible  for  performing  the  task,  and  whether  the  task  was 
coordinated  or  shared  with  the  other  crewman 

b.  Where  training  was  currently  being  conducted 

c.  The  skills  and  knowledges  implicit  in  the  tasks 

d.  The  conditions  under  which  the  tasks  were  to  be  performed,  e.g. , IFR  or  VFR, 
at  night  or  during  daylight 

e.  The  kind  of  cues  available  to  the  crew  members  (e.g. , a feel  or  sound;  a visual 
cue  internal  to  the  cockpit  as  a gage  reading  or  status  light;  a sequence  with  the 
preceding  step  becoming  the  prerequisite  for  the  next) 

f.  The  aircraft  system  or  systems  involved 

g.  The  degree  of  difficulty'  in  performing  the  task 

h.  An  indication  as  to  what  factors  contributed  to  the  task  difficulty,  e.g.,  limited 
time,  lack  of  available  cues,  anxiety  factor,  unfamiliar  situation,  etc. 

i.  The  criticality  of  the  task  in  terms  of  its  impact  if  performed  erroneously  or  not 
at  all,  e.g. , degrade  mission  or  result  in  possible  crash/death 

j.  The  important  performance  measurement  feature  involved,  e.g.,  accuracy, 
precision 

k.  Other  special  factors  which  impact  training,  e.g.,  requires  frequent  practice,  etc. 

To  aid  in  the  gathering  of  this  training  requirements  data,  a form  was  constructed, 
entitled  Task  Analysis  Record  (TAR)  (see  Fig.  3).  With  these  forms,  the  training  and 
educational  specialists  interviewed  the  SMEs  to  obtain  their  judgments  on  the  behavioral, 
conditional,  performance  and  operational  aspects  of  the  task  examined.  One  group  of 
SMEs  aided  in  defining  the  form  content,  both  groups  performed  independent  assessments 
of  the  tasks  under  the  guidance  and  direction  of  the  training  and  educational  specialists. 

The  information  obtained  therefrom  was  used  in  instructional  sequencing  and  blocking 
downstream  in  the  ISD  process,  and  as  an  aid  in  selecting  appropriate  instructional 
strategies. 

A word  about  one  of  the  TAR  items,  skills/knowledges,  seems  appropriate  here.  It 
was  recognized  that  the  analysis  of  the  operational  tasks  identified  in  the  Task  Listing  was 
concerned  not  only  with  the  terminal  behavior  described  by  the  task  statement,  but  also 
with  the  underlying  skills  and  knowledges  which  the  RP  or  RBN  needed  to  have  in  order 
to  successfully  perform  the  task.  A workable  hierarchy  of  skills  and  knowledges  was 
required  which  would  allow  development  of  sound,  workable  Specific  Behavioral  Objectives 
(SBOs).  Lumsdaine,  1 in  1960  indicated  that  the  first  questions  to  be  asked  in  designing 
a training  program  are: 


1 Lumsdaine,  A.  A.,  Design  of  training  aids  and  devices,  in  J.  D.  Folley,  Jr  (ed. ),  Human 
factors  methods  for  a System  Design,  American  Institution  for  Research,  Pittsburgh,  Pa. , 
for  Office  of  Naval  Research,  Report  AIR-C90-60-FR-225,  1960. 
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• What  are  the  final  human  performances  that  should  be  obtained? 

• What  are  the  specific  things  that  need  to  be  learned  to  make  this  criterion 
oerformance  possible? 

• Who  is  to  learn  them?  W'hat  are  his  abilities  and  motivations,  what  part  of  the 
total  required  skills  and  knowledges  does  he  already  have  ? 

In  distinguishing  among  the  different  classes  of  things  trainees  may  need  to  learn, 
Lumsdaine  arrived  at  a five-part  classification  of  training  objectives  which  pose  different 
requirements  for  training  devices  to  satisfy  those  objectives.  The  categorization  of  task 
activities,  labeled  under  the  heading  Task  Components  on  the  Task  Analysis  Form  pro- 
vided in  the  Grumman  proposal  1 2 (see  Fig.  4)  basically  followed  this  classification. 
Additionally,  because  the  task  activities  of  the  B/N  involved  a considerable  amount  of  scope 
reading  and  interpretation  another  category  was  added,  perceptual  discrimination,  as  was 
recommended  by  Parker  and  Downs-.  It  was  found  convenient  to  expand  some  of  the 
knowledge  classifications  later  on  in  preparing  the  behavioral  objectives  and  lesson  speci- 
fications. Here,  a principle  influence  was  Bloom's  Taxonomy  for  cognitive  domain  3. 

The  six  or  seven  classification  taxonomy  used  during  the  gathering  of  the  training 
requirements  data  proved  to  be  appropriate.  Although  by  comparison  to  other  classification 
schemes  examined,  it  might  be  considered  by  some  taxonomists  to  be  "light"  in  the  cognitive 
areas,  it  had  the  advantage  of  being  tailored  to  reflect  the  emphasis  on  performance  in- 
herent in  the  vast  majority  of  pilot  and  B/N  tasks.  It  was  found  fairly  straight-forward  to 
expand  the  knowledge  classification  later  on  in  SBO  development  as  the  need  became  evident, 
as  will  be  described  later  in  this  section. 

Data  Management  System.  A computer  based  data  management  system  was  designed  as  an 
aid  for  the  manipulation  of  data  during  task  and  training  analysis.  The  system  selected 
was  the  commercially  available  MARK  IV  General  Data  Management  system.  The  system  is 
capable  of  sorting  and  assembling  a wide  variety  of  crew  activity  information.  The  system 
operates  on  a task  data  base  which  includes: 

a.  Event,  task,  step  level  activity  descriptors  written  in  natural  language. 

b.  Identification  of  activity  by  aircraft  work  station,  i.e.,  pilot,  B/N,  shared  and 
coordinated. 

c.  Descriptive  data  to  be  used  in  the  training  requirements  analysis  of  task  data, 
e.g.  , task  related  criticality  factors. 


1 op.  cit.  p.  24. 

2 J.  F.  Parker,  J.  E.  Downs,  Selection  of  trair  ing  media.  Aeronautical  Systems 
Division,  Wright- Patterson  A FB,  Ohio,  Report  ASD  TR-61-473.  Sept.  1961. 

3 B.S.  Bloom,  (ed. ),  et  al. , "Taxonomy  of  Educational  Objectives:  The  Classification 
of  Educational  Goals,  Handbook  I:  Cognitive  Domain",  N.Y.:  Longmans,  Green  &•  Co., 
1956. 
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Figure  4.  Task  Analysis  Form  - Proposal 
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The  input  format  of  the  data  system  comprises  a six-card  input.  Cards  No.  2 through 
6 are  assigned  to  the  task  descriptions  in  natural  language.  Card  No.  1 is  reserved  for 
training  analysis  data.  The  computer  program  is  sufficiently  flexible  to  allow  for  a variety 
of  output  reports,  a capability  of  particular  value  during  the  training  analysis  phase.  It 
would  be  highly  useful  to  continue  the  program  development  so  as  to  include  both  SBOs  and 
lesson  data  and  to  relate  these  to  the  task  hierarchy.  Such  a scheme  would  permit  tracing  the 
impact  of  any  aircraft  system  or  hardware  change  down  to  a particular  lesson  within  the 
syllabus.  However  no  funds  were  allocated  for  this  development. 

SBO  and  lesson  specification  data  consequently  were  manipulated  manually.  This  lack 
of  electronic  data  processing  did  not  seriously  impact  the  job  because  only  two  job  descrip- 
tions, pilot  and  B/N  were  being  analyzed.  Such  support  would  be  almost  imperative  for  per- 
forming the  ISD  process  for  operations  involving  multiple  assignments,  e.g. , shipboard  oper- 
ations or  multiple  activities,  e.g. , maintenance  operations.  Further,  completion  of  the  pro- 
gram might  provide  a convenient  tool  for  Navy  updating  of  the  training  program  once  it  is  in 
operation. 

SPECIFIC  BEHAVIORAL  OBJECTIVES  (SBOs) 

INTRODUCTION.  In  designing  an  Instructional  System,  the  next  step  which  was  taken  was  the 
development  of  task  related  specific  behavioral  objectives.  In  the  past,  training  curricula 
were  often  developed  around  lofty  and  vague  goals,  and  learning  outcomes  often  lost  the  nec- 
essary relationship  to  the  operational  tasks.  The  utilization  of  SBOs  in  establishing  training 
curricula  virtually  eliminates  this  undesirable  result.  The  development  of  SBOs  began  with 
the  establishment  of  major  categories  for  a Taxonomy  of  Training  Objectives.  This  taxonomy 
was  to  be  an  important  tool  in  the  next  two  activities  of  the  process,  the  first  of  which  was 
completing  the  Task  Analysis  Record  sheets.  The  contractor,  with  the  aid  of  the  VA-42  and 
VA-128  Subject  Matter  Experts  (SMEs),  reviewed  the  pilot  and  B/N  Task  Listings  and 
dcoumented,  on  the  Task  Analysis  Record  Sheets,  selected  information,  i.e.  , crewmember, 
conditions,  system  involved  which  would  be  valuable  in  developing  the  SBOs.  To  develop  the 
SBOs,  all  the  high  level  job  related  behaviors  had  to  be  identified  first,  i.e.  , behaviors  that 
repeat  or  closely  resemble  the  behavior  required  in  performing  the  task.  The  task  listings 
were  exhaustively  analyzed,  task  by  task  and  step  by  step.  The  behaviors  generated  consti- 
tuted the  heart  of  the  Terminal  Objectives  (TOs). 

The  high  level  behaviors  were  analyzed  next,  to  identify  the  enabling  behaviors.  These 
behaviors  are  the  prerequisites  to  the  performance  of  the  job  related  behaviors  and  became 
the  basis  for  the  Enabling  Objectives  (EOs).  This  behavioral  information,  together  with  the 
information  gathered  on  the  Task  Analysis  Record  Sheets,  formed  the  groundwork  for  the 
construction  of  the  SBOs  and  the  formation  of  an  objectives  hierarchy,  i.e.  , a sequence  of 
EOs  supporting  a TO.  Of  the  373  SBOs  identified,  105  were  Pilot  related,  109  B/N  related 
and  159  involved  both  crewmen.  The  SBOs  were  documented  in  three  separate  volumes  1-3 
and  submitted  to  Naval  Training  Equipment  Center  for  review  and  approval. 

1 S.  Campbell,  G.  Graham  and  S.  Hersh,  Specific  Behavioral  Objectives  and  Criterion  Test 
Statements,  Pilot  Procedures  A-6E  (TRAM)  Aircraft  (Vol.  I),  Grumman  Aerospace  Corp. , 
Bethpage,  N.Y.,  Report  ISD  (SAT)  0003— 1,  December  1975. 

2 J.  Hanish  and  J.  Feddern,  Specific  Behavioral  Objectives  and  Criterion  Test  Statements, 
Bombardier/Navigator  Procedures,  A-fiE  (TRAM)  Aircraft,  Volume  II,  Grumman  Aero- 
space Corp.,  Bethpage,  N.Y.,  Report  ISD  (SAT)  0003-5,  Dec.  1975. 

3 G.  Graham,  J.  Hanish  and  S.  Hersh,  Specific  Behavioral  Objectives  and  Criterion  Test 
Statement,  Airframe  Emergencies  andSystem  Malfunctions,  A-0E  (TRAM)  Aircraft  (Vol- 
ume III)  Grumman  Aerospace  Corp.,  Bethpage,  N.Y.,  Report  ISD  (SAT)  0003-fi,  Dec.  1975. 
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PROCESS.  The  initial  activity  in  developing  SBOs  was  the  establishment  of  a suitable  taxo- 
nomy of  training  objectives.  To  this  end,  the  ISD  Group  reviewed  taxonomies  developed  by 
Bloom1,  Lumsdaine2,  Krathwohl3  and  the  Department  of  the  Air  Force4.  The  taxonomies 
developed  by  Bloom  and  Krathwhol  were  heavily  slanted  toward  the  cognitive  and  the  affective 
domains  while  the  Air  Force  taxonomy  leaned  toward  the  psychomotor  domain.  Some  con- 
sideration was  given  to  integrating  the  categories  of  all  of  the  taxonomies;  however,  the 
resultant  large  number  of  categories  that  would  have  required  examination  for  each  behavior 
would  have  led  to  delay  and  confusion.  It  was  decided  to  limit  the  number  of  categories  in 
the  taxonomy  by  judiciously  selecting  what  was  considered  knowledge  and  skill  categories 
pertinent  to  the  immediate  task.  In  addition,  the  elimination  of  the  affective  domain  cate- 
gories was  made  possible  by  recognizing  that  the  students  are  replacement  pilots  and  B/Ns 
who  are  carefully  selected,  highly  motivated  and  painstakingly  screened  in  prior  training 
programs.  This  resulted  in  the  establishment  of  a taxonomy  of  training  objectives  specific 
to  flight  crew  operations  (Table  3). 

The  second  step  of  the  process  involved  performing  an  analysis  to  identify  task  related 
training  factors  i.e. , the  subject  of  the  tasks,  the  skills  and  knowledges  required,  the  con- 
ditions under  which  the  task  will  be  performed,  potential  cues,  the  systems  exercised,  the 
degree  of  difficulty  associated  with  performing  the  task,  the  criticality  of  the  task,  the  per- 
formance measurements  and  the  special  considerations  involved.  A list  of  potential  train- 
ing factors  was  identified  and  documented  in  a standard  form,  the  Task  Analysis  Record  Sheet 
(Figure  3).  The  correlation  of  these  training  factors  to  Pilot  and  B/N  tasks  was  a team  effort, 
involving  Navy  SMEs  from  VA-42  and  VA-128  as  well  as  contractor  Educational  Specialists. 
The  SMEs  provided  the  invaluable  aspects  of  experience  and  operational  expertise  needed  to 
relate  each  training  factor  to  a task  operation.  This  relationship  was  permanently  recorded 
on  the  Task  Analysis  Record  Sheets  and  in  the  Data  Management  computer  file  for  future  use 


• Developing  SBOs 

• Blocking  and  sequencing  SBOs 


op.  cit. , p.  43. 
op.  cit. , p.  41. 

Krathwohl,  D.  R. , Taxonomy  of  Educational  Objectives  - Its  Use  in  Curriculum  Build- 
ing and  Defining  Educational  Objectives,  University  of  Pittsburgh,  Pa.,  1964. 

AFP  50-58,  op.  cit.  , p.  12. 
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TABLE  3.  TAXONOMY  OF  TRAINING  OBJECTIVES 


1 . 0 Knowledge 

1.1  Technological  - Learning  "how  to"  perform  a single  switch  and  control  con- 
figuring procedure.  Learning  "how  to"  read  meters,  digital  displays,  scopes, 
lighting  displays,  etc.  In  general,  learning  "how  to". 

1.2  Formal  - Learning  the  meaning  of  special  symbols,  acronyms,  words,  nomen- 
clature, etc. 

1.3  Descriptive  - To  describe  "what  is"  and  "what  was":  facts,  data,  special  infor- 
mation about  systems,  subsystems,  equipment,  weapons,  tactics,  missions, 
etc. 

1.4  Concepts  and  Principles  - Fundamental  truths,  ideas,  opinions  and  thoughts 
formed  from  generalizations  of  particulars. 

2.0  Comprehension 

Understanding  the  meaning  of  meter  readings,  scope,  digital  and  lighting  displays. 
Understanding  the  switch  and  control  configuring  procedure,  i.e.,  the  reason  for  a 
specific  sequence,  the  reason  for  a switch  or  control  position,  the  reason  for  a veri- 
fication, etc. 

Grasping  the  meaning  of  concepts  and  principles,  i.e. , understanding  the  basic 
principles  of  infrared  and  radar  detection. 

Understanding  the  meaning  of  facts,  data,  specific  information,  etc. 

3.0  Discrimination 

Distinguishing  among  different  external  stimuli  and  making  appropriate  responses  to 
them,  e.g.,  scanning  gages  for  out-of-tolerance  trends.  Also  includes  the  recognition 
of  the  essential  similarity  among  a class  of  objects  or  events,  e.g. , classifying  air- 
craft types  or  radar  return  images. 

4.0  Application 

4.1  Simple  Procedure  - A demonstration  of  a simple  learned  procedure  in  the  cock- 
pit or  simulator  requiring  not  more  than  simply  repeating  required  switch  and 
control  configuring  and  simple  visual  verification,  i.e.  , advisory  light  status. 

4.2  Complex  Procedure  - A demostralion  of  a learned  procedure  in  a cockpit  or 
simulator  that  requires  differentiating  or  distinguishing  between  readings  on 
meters,  digital  displays,  and  images  on  video  and  radar  displays  and  inter- 
preting and  applying  the  meaning  of  the  readings  and  images. 


4.3  General  - Using  learned  materials  in  new  and  concrete  situations,  e.g.  , using 
rules,  methods,  concepts,  principles,  procedures,  etc. 
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TABLE  3.  TAXONOMY  OF  TRAINING  OBJECTIVES  (CONT'D' 

5.0  Analysis 

A demonstration  of  a learned  process  of  breaking  down  material  i.  e.  , data,  other 
information  into  its  components  so  that  it  may  be  evaluated  with  respect  to  crew 
safety,  mission  success,  A/C  maintenance,  etc. 

6.0  Synthesis 

A demonstration  of  a learned  process,  e.g. , putting  tactical  elements  together 
weapons,  targets,  available  systems,  A/C  capability,  etc.)  to  formulate  a mission. 

7.0  Evaluation 

A demonstration  of  a learned  process  of  assessing  or  judging  a system  or  situation, 
based  on  criteria  i.  e. , data,  rules,  available  equipment,  conditions,  etc.  and  then 
reaching  a conclusion  based  on  this  assessment. 

8.0  Complex  Performance 

A demonstration  that  requires  psychomotor  skills  and/or  critical  thinking  skills 
usually  requiring  practice. 

In  step  three  of  the  process,  the  characteristics  of  the  Specific  Behavioral  Objective 
were  identified.  Contractor  Educational  Specialists  reviewed  the  works  of  such  authorities 
in  the  field  as  Robert  R.  Mager1,  Robert  M.  Cagne2,  David  R.  Krathwohl3,  the  Department 
of  the  Air  Force4  and  the  U.S.  Training  and  Doctrine  Command5,  and  concluded  that  they 
all  essentially  agreed  on  the  content  of  an  SBO.  A well-prepared  SBO  contains  the 
following: 

a.  A description  of  the  learner 

b.  A specific  description  of  the  behavior,  expressed  as  an  action,  that  the  learner 
is  to  exhibit  after  training 


1 R.  R.  Mager,  Preparing  Objectives  for  Programmed  Instructions,  San  Francisco,  Ca., 
Fearon  Publishers,  1962. 

2 R.  M.  Gagne,  The  Analysis  of  Instructional  Objectives,  Paper  for  National  Educational 
Association  (NEA),  1963. 

3 Krathwohl,  op.  cit. , p.  46. 

4 Department  of  Air  Force  ATP  50-58,  op.  cit.  ,p.  12. 

® U.S.  Training  and  Doctrine  Command,  Interservice  Procedures  for  Instructional 

Systems  Development,  Vols.  I-V,  Center  for  Educational  Technology,  Florida  State  U. , 
for  U.S.  Training  and  Doctrine  Command,  Report  TRADOC  Pamphlet  350-30,  Aug.  1975. 
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c.  The  conditions  under  which  the  action  will  take  place 


d.  The  minimum  acceptable  performance  standards  under  which  the  desired  behavior 
is  to  be  exhibited. 

Two  classifications  of  SBOs  were  identified:  terminal  and  enabling.  The  Terminal 
Objective  (TO)  is  the  highest  level  of  behavioral  change  that  can  be  associated  with  a task 
and  is  usually  directly  job  related,  e.g. , "The  RBN  will  copy  and  analyze  all  frozen  data 
from  the  DDU  and  DVRI  without  omission  or  error. " The  Enabling  Objective  (EO)  is  the 
lower  level  behavioral  change  or,  prerequisite  that  must  be  achieved  prior  to  being 
able  to  perform  the  TO,  e.g. , the  student  must  understand  the  relationship  between  the 
DDU  and  the  DVRI  frozen  data  and  the  ensuing  impact  error  analysis  prior  to  analyzing 
the  data.  Experience  indicates  that  there  is  usually  at  least  one  EO  for  each  TO. 

The  next  step  was  to  develop  a standard  report  format  for  recording  identified  SBOs. 
The  result  was  the  Instructional  Systems  Design  Record  (Fig.  5)  which  contains  the  follow- 
ing information: 

a.  Task  Listing  - Reference  to  specific  volume  and  page  of  the  A-6E  TRAM  Task 
Listing. 

b.  Task  Number  - Specific  event  (E),  Task  (T)  or  Step  (S),  from  that  Task  Listing, 
for  which  the  SBO  has  been  written. 

c.  Task  Responsibility  - Indicates  prime  responsibility  for  the  task,  pilot,  B/N, 
coordinated  or  shared  (both). 

d.  Task  Statement  - A statement  of  the  operational  activity  expected  of  the  pilot 
and/or  B/N. 

e.  Conditions/Constraints  - Statement  of  the  operational  conditions  or  operational 
constraints  against  which  the  tasks  might  need  to  be  performed.  Where  this  seg- 
ment of  the  form  is  left  blank,  e.g. , for  many  conditional  performance  activities 
(malfunctions),  no  qualifying  conditions  or  constraints,  which  might  differentially 
affect  the  task  performance,  have  been  identified. 

f.  Performance  Standard  - The  standard  against  which  the  operational  performance 
is  judged.  Meeting  this  standard  in  the  fleet  affirms  that  the  task  activity  has 
been  satisfactorily  performed.  The  standards  indicated  reflect  the  consensus 
arrived  at  during  working  sessions  with  VA-42  and  VA-128  SME  personnel. 

g.  SBO  Statement  - The  learning  outcome  expected  to  be  achieved  and  exhibited  at 
the  conclusion  of  training  in  the  Readiness  Squadron.  SBO  statements  may  be 
classified  as  Terminal,  the  final  behavior  to  be  exhibited,  or  Enabling,  a learn- 
ing outcome  prerequisite  to  a terminal  behavior. 

h.  SBO  Number  - A unique  identifying  number  for  each  SBO. 

i.  Skill/Knowledge  Class  - A hierarchial  classification  of  skills  and  knowledges  im- 
plied in  the  performance  of  the  specified  behaviors.  May  range  from  simple 
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knowledge  through  the  higher  order  mental  functions  of  analysis  or  evaluation,  or 
from  simple  procedural  responses  to  complex  psychomotor  performance.  This 
block  is  intended  for  use  during  development  of  instructional  strategies,  sequenc- 
ing of  instructional  blocks,  and  formation  of  lesson  specifications. 

j.  Criterion  Test  Statement  - A statement  of  the  conditions  under  which  the  SBO 
should  be  performed  or  demonstrated.  It  also  includes  the  associated  standard 
of  performance  which  must  be  achieved  in  order  to  ensure  the  SBO  behavior  when 
in  the  operational  situation.  Whereas  the  SBO  statement  implies  1007t  standard 
of  performance  (in  a single  trial)  the  criterion  test  demands  repeatability  of  be- 
havior at  a certain  level  as  demonstrative  that  behavior  is  indeed  "learned". 

k.  CT  Number  - Similar  to  the  SBO  number,  a unique  numerical  identifier. 

l.  Test  Type  - A preliminary  and  general  classification  of  the  suggested  form  which 
the  criterion  test  might  take.  Typically  subdivided  into  either  written,  oral  or 
demonstration  tests,  conducted  in  a classroom,  in  the  A/C  or  in  a simulator. 

More  definitive  recommendations  are  made  following  completion  of  the  Media 
Analysis. 

m.  Test  Format  - This  section  of  the  SBO/CT  form  is  completed  as  part  of  the 
Media  Analysis  and  used  during  sequencing  of  instructional  blocks. 

As  one  looks  at  the  record  sheet,  it  becomes  obvious  that  it  deviates  slightly  from 
the  conventional  method  of  structuring  an  SBO.  The  Criterion  Test  Statement  has  been 
deliberately  separated  from  the  SBO  statement.  This  was  done  to  accommodate  the  fact 
that  a performance  measurement  could  not  always  be  accomplished  under  operational  con- 
ditions/constraints and  to  the  operational  standards.  For  example,  the  performance  of  a 
malfunction  procedure  in  an  operational  environment  and  to  operational  standards  is  just  not 
feasible  in  an  A/C.  Therefore,  this  type  of  performance  measurement  is  conducted  in  a 
WST  where  the  malfunction  can  be  simulated  and  the  performance  evaluated  to  the  standards 
dictated  by  the  capacity  or  limitations  of  the  simulation.  Another  consideration  was  the  con- 
fidence level  instilled  in  the  instructor  as  a result  of  a student  performing  a task  only  once 
to  the  operational  performance  standards.  For  the  instructor  to  have  confidence  that  the 
student  will  meet  or  exceed  the  minimum  operational  standards  in  the  fleet,  he  must  witness 
that  the  student  has  met  or  exceeded  the  standards  considered  appropriate  at  the  Readiness 
Squadron  level  of  training  a bonafide  number  of  times  during  the  criterion  test.  So,  although 
the  format  deviates  slightly  from  the  standard  method  of  writing  the  SBOs,  it  presents  all 
the  characteristics  and  still  allowed  enough  flexibility  to  incorporate  learning  conditions, 
constraints,  and  performance  standards  where  necessary. 

In  the  next  step  of  the  process,  pilot  and  B/N  job-related  behaviors  were  specified  as 
a result  of  an  in-depth  analysis  of  the  Task  Listing  and  then  integrated  with  the  related 
training  factors  previously  identified  in  the  Task  Analysis  to  formulate  SBOs.  The  high- 
level  behaviors  i.  e.  , behaviors  that  replicate  or  closely  resemble  the  task  were  identified 
first  by  analyzing  the  activities  task  by  task  and  step  by  step.  These  behaviors  formed  the 
substance  of  subsequently  constructed  TOs.  In  turn,  these  high-level  behaviors  were  anal- 
yzed to  identify  prerequisite  behaviors  which  must  be  learned  prior  to  being  able  to  exhibit 
a high-level  behavior.  These  behaviors  constituted  the  substance  of  subsequently  constructed 
EOs.  At  this  time,  a previously  recognized  problem,  lack  of  documented  standards,  had  to 
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be  resolved.  Realizing  that  the  terminal  objectives  from  prerequisite  training  programs 
were  not  available  in  accessible  documented  form,  the  SME  team  members  agreed  to 
identify,  as  best  they  could,  the  entry  levels  for  the  Readiness  Squadron  pilot  and  B/N. 
Immediately  after  the  job  related  behaviors  were  identified  for  a particular  task  of  the  Task 
Listing,  they  were  combined  with  the  appropriate  information  gathered  on  the  Task  Analysis 
Record  Sheet  to  formulate  terminal  and  enabling  SBOs  and  associated  criterion  tests.  All 
of  this  SBO  data  was  recorded  on  the  Instructional  Systems  Design  Record  Sheets  and  placed 
in  an  objective  hierarchy.  In  all,  373  SBOs  were  identified;  109  were  identified  for  the 
B/N,  105  for  the  Pilot  and  159  for  the  Pilot  and  B/N.  Once  the  SBOs  were  developed  and 
recorded,  a review  was  conducted  by  VA-42  and  VA-128.  Their  comments  were  primarily 
concerned  with  relaxing  the  performance  standards.  They  did  not  provide  any  negative 
comments  with  respect  to  the  adequacy  or  applicability  of  the  SBOs.  Following  this  review 
the  SBOs  were  used  in  the  development  of  the  Lesson  Specifications. 

MEDIA  ANALYSIS 

INTRODUCTION.  Media  selection  is  the  principal  factor  in  determining  how  instruction  is 
to  be  packaged  and  presented  to  the  student.  The  selection  of  media  affects  both  the  cost  and 
effectiveness  of  training.  The  first  consideration  relative  to  media  is  whether  or  not  any 
media  are  required.  Ideally  the  A-G  pilot  and  B/N  could  be  completely  trained  on-the-job, 
with  only  the  aircraft  and  operational  facilities  and  documents  as  aids.  A brief  examination 
of  the  reality  of  the  situation  reveals  that  this  can  not  be  the  case.  There  are  practical  con- 
siderations related  to  cost,  safety,  and  the  availability  of  aircraft  that  constrain  the  exclusive 
use  of  operational  resources  in  the  aircrew  training  program.  There  are  also  instructional 
considerations  such  as  ensuring  student  proficiency  in  emergency/malfunction  situations, 
most  of  which  would  not  occur  in  the  "normal"  environment. 

Therefore,  alternatives  to  the  operational  aircraft  must  be  considered.  A systematic 
approach  to  the  Media  Analysis  required  a consideration  of  the  following  factors  related  to 
the  objectives:  the  skill/knowledge  category  of  each  group  of  objectives,  the  instructional 
requirements  of  each  objective,  an  understanding  of  the  stimulus  characteristics  of  the 
various  media  alternatives,  a selection  of  media  mixes  that  have  the  needed  characteristics, 
and  finally  an  analysis  of  the  cost  factors  associated  with  the  media  alternatives. 

THE  PROCESS.  The  media  analysis  began  with  an  examination  of  the  Specific  Behavioral  Ob- 
jectives (SBOs),  at  the  terminal  and  particularly  at  the  enabling  level.  This  examination  had 
the  purpose  of  identifying  the  various  activities  and  kinds  of  learning  involved,  such  that  ap- 
propriate media  satisfying  the  characteristics  of  that  learning  might  be  identified.  During 
the  examination  it  became  obvious  that  the  SBOs  as  written  often  required  a lower,  more  de- 
tailed level  of  treatment  in  order  to  ascertain  appropriate  media.  These  more  detailed 
statements  were  entitled  Learning  Points.  These  learning  points  were  merely  another  cut 
in  the  tier  of  enabling  objectives,  and  were  so  converted  and  labeled  when  the  lesson 
specifications  were  prepared. 
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A review  of  ISD  reference  documents1-1  revealed  a commonality  of  approach  in  the 
identification  of  media  characteristics.  The  various  forms  differ,  as  do  the  number  of 
variables,  but  the  process  is  essentially  the  same.  The  primary  reference  during  the  Media 
Analysis  phase  has  been  the  work  of  Boucher,  Gottlieb  and  Morganlander,  Handbook  and 
Catalog  tor  Instructional  Media  Selection*1. 

A preliminary  step  in  the  media  selection  process  involved  the  analysis  of  objectives/ 
learning  points  to  determine  the  information  to  be  provided  the  learner  and  the  behavior  ex- 
pected of  him.  This  wras  done  by  considering  the  media  requirements  for  three  phases  for 
each  objective/learning  point;  (1)  Initial  Instruction  (2)  Practice;  and  (3)  Demonstration/ 
Test. 

The  following  questions  were  then  asked,  with  respect  to  each  phase  separately: 

(1)  what  are  the  essential  stimulus  characteristics  necessary  to  present  the  material;  and 

(2)  what  capabilities  are  required  for  optimum  student  response?  Initially,  these  questions 
were  addressed  using  the  Media  Capabilities  Work  Sheet  from  Boucher,  et  al.  (see  Fig.  fi). 

It  became  apparent,  however,  that  the  "universal  application"  nature  of  this  worksheet 
caused  a number  of  considerations  to  be  addressed  that  were  not  appropriate  for  this  stage 
of  the  study.  These  considerations,  premature  in  nature,  were  found  primarily  in  the 
Instructional  Strategy  section  of  the  worksheet.  It  was  also  found  that  some  of  the  presen- 
tation characteristics  were  not  relevant  to  this  study.  For  these  reasons  the  worksheet  was 
first  revised,  and  then  put  aside  in  favor  of  a simplified  Media  Analysis  Table  (see  Table  4). 

In  the  Media  Analysis  Table,  SBOs  (left-hand  column)  possessing  similar  learning 
outcomes  have  been  grouped  together  (a  first  step  in  the  blocking  process).  Each  group  of 
one  or  more  SBOs  is  separated  by  a horizontal  line  across  the  page.  Next  to  each  SBO 
group  are  one  or  more  learnirg  points  associated  with  all  of  the  SBOs  of  the  group.  To  the 
right  of  each  learning  point  are  the  media  devices  selected  for  presentation  and  use  for  the 
three  learning  phases  (initial  learning,  practice,  demonstration/test),  to  meet  the  educa- 
tional requirements  of  the  learning  point  and  in  turn,  the  SBOs  in  the  group.  It  should  be 
noted  that  the  mental  process  of  assigning  media  to  each  of  these  phases  was  a continuation 
of  the  thought  processes  begun  with  the  Media  Capabilities  Worksheet  (Fig.  6)  although  the 
worksheet  itself  was  no  longer  used,  hi  the  column  labeled  Initial  Learning  are  the  media 
devices  used  for  presentation  of  material  to  the  student  as  required  by  the  learning  points. 
The  devices  used  to  reinforce  and  practice  the  newly  learned  skills  are  listed  in  the  column 
labeled  Practice.  The  entries  in  the  Demonstration/Test  column  are  the  media  used  for  the 
criterion  test  of  the  learning  point. 

Media  Classification  and  Results.  The  summary  tables  showing  the  results  of  the  media 
analysis  have  been  organized  by  categories  much  in  accord  with  the  classification  grouping 
shown  in  AFP  50-58,  Handbook  for  Design  of  Instructional  Systems,  with  the  notable 


Dept,  of  Air  Force,  AFM  50-2  and  AFP  50-58,  op.  cit.  p.  12. 

2 Training  Analysis  and  Evaluation  Group,  An  Evaluation  of  Ten  Techniques  for  Choosing 
Instructional  Media,  Orlando,  Fla.,  Report  TA EG  No.  8,  Dec.  1973. 

2 U.  S.  Training  & Doctrine  Command,  op.  cit.  p.  18. 

1 B.  G.  Boucher,  M.  J.  Gottlieb  and  M.  L.  .Morgan lander,  Handbook  and  Catalog  for 
Instructional  Media  Selection,  Educational  Technology  Publications,  Englewood  Cliffs, 
N.J. , 1973. 
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differences  given  in  the  following  category  breakdown.  Only  media  for  which  a requirement 
was  identified  are  shown  in  the  tables.  Thus,  whole  and  part  task  trainers,  opaque  projec- 
tions, teaching  machines,  and  other  media  from  AFP  50-58  are  not  illustrated. 

a.  Graphics  Media  Category  - Inclusions  in  this  category  have  been  restricted  to  in- 
structor employed  aids,  such  as,  wall  charts,  oversized  system  diagrams  and  cockpit  lay- 
out drawings.  Other  graphic  materials  have  been  included  either  under  the  audio  visual 
category,  if  projected,  or  within  the  printed  media  category,  when  student  utilized,  e.g. , 
photographs,  illustrations/drawings,  charts,  graphs,  checklists,  etc. 

b.  Printed  Media  Category  - Although  separately  identified  in  the  media  analysis 
tables  found  in  the  media  analysis  document  1 , and  relating  media  to  SBO  groups,  it  has 
been  excluded  from  the  summary  tables  contained  herein.  It  is  assumed  that  this  category 
of  media  will  be  available  as  required  e.g. , NATOPS  FAM  and  TAC  manuals,  checklists, 
lesson/study  guides,  etc. 

c.  Tutorial  Media  Category  - Carrels  have  been  added  to  this  category. 

d.  Environmental  Media/Aids  Category  - A specific  media  subset,  encompassing 
features  of  both  games  and  role  playing  aids  of  AFP  50-58,  called  paper  simulation,  has 
been  identified  as  a useful  tool  for  "headwork"  assessment  of  mission  planning  and  in-flight 
mission  decision-making.  Paper  simulation  is  discussed  further  in  the  paragraphs  provid- 
ing a functional  description  of  the  candidate  devices  found  in  this  report. 

e.  Operational  Equipment  Category  - This  category,  which  includes  the  A-6E  aircraft 
and  the  TC-4C  trainer  has  been  added  to  the  media  classification. 

Tables  5 and  6 provide  summaries  of  the  media  analysis.  Tables  containing  the  de- 
tailed media  analysis  data  are  found  in  the  Appendix  section  of  the  Media  Analysis  Report. 
Table  5,  SBO  count  per  media  type,  indicates  the  number  of  SBOs  in  the  training  program 
for  which  the  use  of  different  types  of  media  is  recommended.  Volume  I refers  to  SBOs 
identified  for  pilot  training  for  normal  operations;  Volume  II  refers  to  B/N  normal  opera- 
tions; and  Volume  III  refers  to  Pilot  and  B/N  Emergency  Procedures  and  Weapon  System 
Failures.  In  the  main,  the  same  media  are  used  to  the  same  extent  for  both  pilot  and  B/N 
training.  The  principal  items  are  sound  slide  projections  and  overhead  transparencies, 

WST  and  Procedures  Trainers.  Pilot  training  SBOs  reflect  a higher  proportion  of  the  use  of 
the  A/C  as  a medium  than  do  the  B/N  SBOs.  These  in  turn  employ  the  TC-4C  trainer  for 
certain  weapons  system  operations  which  cannot  be  trained  in  the  A/C  or  WST  because  of 
instructional  considerations  or  capability  limitations.  Since  the  A-6E  TRAM  ISD  program 
emphasizes  operational  considerations,  the  use  of  simulated  or  operational  equipment  is 
highly  desirable,  and  is  so  reflected  in  the  media  recommended. 

Since  the  weapon  system  operations  involve  a considerable  amount  of  radar  scope  in- 
terpretation and  FLIR  imagery  changes,  the  use  of  16  mm  film  and  video  tape  as  a training 
media  tool  is  emphasized  for  B/N  training. 

Table  6 summarizes  the  use  of  media,  again  related  to  SBOs,  for  three  phases  of 
training:  the  initial  learning  phase  (IL)  which  includes  much  of  the  basic  familiarization 
training  as  well  as  some  skill  development,  the  practice  phase  (P)  in  which  skills  and 
knowledges  are  refined  and  improved,  and  the  demonstration  or  test  phase  (D/T). 


Feddem,  et  al. , Report  ISD  (SAT)  0004,  op.  cit,  p.  31. 
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TABLE  5.  SUMMARY  OF  SBO  COUNT  PER  MEDIA  TYPE 


Media 

Volume  I 

Volume  II 

Volume  III 

Total  SBOs 

Transient  Media 

• Audio  Visual 
-sound  slide 
projector 

52 

56 

58 

166 

-live  TV 

4 

0 

0 

4 

-video  tape 

36 

49 

4 

89 

-16  mm  film 

3 

49 

12 

64 

Visual  Aids 

• Graphics 
-wall  charts 

>0 

47 

0 

57 

• Exhibits 

-mockups/models 

5 

0 

0 

5 

• Projected  Images 
-overhead 
transparencies 

67 

53 

58 

178 

-slides 

0 

12 

0 

12 

Tutorial  Media 

• Carrel 

30 

66 

43 

139 

Environmental 

Media/Aids 

• WST 

71 

75 

89 

235 

• Procedures 
Trainer 

37 

48 

43 

128 

• Paper  Simulation 

* 

* 

* 

Operational 

Equipment 

A/C 

92 

58 

11 

161 

TC-4C 

0 

15 

0 

15 
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TABLE  6.  SUMMARY  OF  MEDIA  COUNT  BY  INSTRUCTIONAL  PHASE 


Media 

Volume  I 
IL  P D/T 

Volume  n 
IL  P D/T 

Volume  III 
IL  P D/T 

IL 

Total 

P 

D/T 

Transient  Media 

• Audio  Visual 
-sound  slide 
projector 

53 

3 

1 

51 

24 

0 

58 

0 

0 

162 

27 

1 

-live  TV 

4 

0 

0 

0 

0 

0 

0 

0 

0 

4 

0 

0 

-video  tape 

38 

1 

1 

49 

12 

0 

4 

0 

0 

91 

13 

1 

-16  mm  film 

3 

0 

0 

49 

0 

0 

0 

0 

12 

52 

0 

12 

Visual  Aids 

• Graphics 
-wall  charts 

10 

0 

0 

47 

0 

0 

0 

0 

0 

57 

0 

0 

• Exhibits 

-mockups/models 

4 

0 

1 

0 

0 

0 

0 

0 

0 

4 

0 

1 

• Projected  Images 
-overhead 
transparencies 

67 

2 

1 

53 

0 

1 

58 

0 

0 

178 

2 

2 

-slides 

0 

0 

0 

0 

12 

0 

0 

0 

0 

0 

12 

0 

Tutorial  Media 

• Carrel 

31 

2 

0 

40 

46 

0 

44 

0 

0 

115 

48 

0 

Environmental 

Media/Aids 

• WST 

17 

68 

12 

5 

74 

51 

11 

90 

82 

33 

232 

145 

• Procedures 
Trainer 

3 

37 

0 

0 

48 

0 

22 

43 

0 

25 

128 

0 

• Paper  Simulation 

♦ 

* 

* 

* 

Operational 

Equipment 

A/C 

11 

69 

83 

0 

33 

58 

0 

11 

0 

11 

102 

141 

TC-4C 

0 

0 

0 

0 

11 

14 

0 

0 

0 

0 

11 

14 
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Here  again,  the  media  recommended  for  both  pilot  and  B/N  training  are  almost  identical. 

As  would  be  expected,  a dramatic  transition  from  use  of  audio  visual  and  other  visual  aid 
media  in  the  initial  learning,  to  the  environmental  media  and  operational  equipment  during 
the  practice  phase,  is  indicated.  Because  of  the  fact  that  the  A-6E  weapon  system  is  new 
to  the  replacement  B/N,  considerable  individual  practice  using  a carrel  is  seen  in  the  initial 
learning  phase  and  continuing  into  the  practice  phase  of  training.  The  procedures  trainer  is 
recommended  to  support  some  initial  learning/training  in  nominal  and  emergency  procedures, 
but  is  used  predominantly  for  practice  of  airframe  emergency  procedures  for  the  pilot  and  for 
off-nominal  or  alternate  weapons  system  procedures  for  the  B/N.  It  is  not  recommended  for 
use  in  the  demonstration  or  test  phase  of  training. 

Functional  Description  of  Candidate  Devices.  The  media  analysis  resulted  in  the  identifica- 
tion of  a number  of  candidate  devices  which  will  provide  the  training  support  desired  in 
satisfying  the  totality  of  training  objectives.  Candidate  device  selection  followed  upon  com- 
pletion of  a survey  of  devices  in  inventory,  particularly  those  units  available  within  VA-42 
and  VA-128,  Readiness  training  squadrons,  and  those  which  might  be  obtained  from  audio- 
visual manufacturers.  Findings  of  the  field  site  survey  are  reported  later  in  this  section. 

Further,  the  survey  continued  with  an  analysis  of  the  A-6E  Weapons  System  Trainer 
(WST)  and  the  A-6  Suitcase  Emergency  Procedures  Trainer  (SCEPTR)  with  regard  to  existing 
configurations  and  desired  modification^. 

Nine  categories  or  classifications  of  media  devices  were  identified  and  recommended 
for  use  in  the  training  program.  They  are: 

• Procedures  Trainers  (PT) 

• Paper  Simulation 

• Sound-Slide  Devices 

• Video  Tape  Player  Devices 

• 10mm  Motion  Picture  Devices 

• Overhead  Transparency  Projection  Devices 

• Learning  Carrels 

• Weapon  System  Trainer  (WST) 

• TC-4C  Inflight  Trainer 

In  addition  the  use  of  the  A-6E  aircraft  in  a training  mode  is  considered  a primary  element 
of  the  training  program. 

The  media  analysis  report1  includes  a series  of  data  sheets  providing  generic, 
categorical  descriptions  of  the  audio  visual  devices,  functional  descriptions  and  pricing 


1 J.  Feddern,  etal..  Report  IS D (SAT)  0004,  op.  cit. , p.  31. 
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data  for  specific  items.  The  data  sheets  for  the  remaining  media  devices  (PT,  WST,  TC-4C 
and  paper  simulations)  provide  functional  descriptions  with  preliminary  pricing  information 
indicated  for  the  B/N  Procedures  Trainer.  They  are  considered  sufficiently  important  to 
an  appreciation  of  the  conclusion  drawn  from  the  media  study  to  include  the  essentials  of 
those  data  sheets  herein. 

Procedures  Trainers.  Reference  to  the  summary  tables  (Tables  5 and  6)  shows  that  there 
are  at  least  153  instances  where  the  intersection  of  B/N  or  pilot  teaching  strategies  and 
SBOs  warrant  early  procedures  training.  Presently  the  only  device  planned  or  available 
that  can  satisfy  these  requirements  is  the  WST.  In  consideration  of  the  hi-fidelity  flight 
simulation  capability  of  the  WST  and  its  intended  use  in  the  overall  A-6  flight  crew  training 
program,  application  of  this  device  for  initial  procedural  familiarization,  either  nominal  or 
emergency,  or  early  development  of  cognitive  skills  appears  to  be  an  Inefficient  utilization 
of  this  resource. 

Consequently  it  is  recommended  that  both  a B/N  Procedures  Trainer  and  a Pilot  Pro- 
cedures Trainer  be  included  in  the  media  inventory  to  fill  the  gap  between  classroom  in- 
struction and  hi-fidelity  integrated  weapons  system  training  or  pilot  flight  related  activities. 
The  Trainer  would  be  used  for  reinforcement  of  concepts  of  operation,  initial  development 
of  related  display  and  control  operational  skills,  and  procedural  drills  involving  assessment 
of  airframe  emergencies,  assessment  of  weapons  system  configurations  and  subsequent 
selection  of  optimum  alternate  or  secondary  paths  for  off-nominal  conditions.  The  latter 
application  involves  early  development  of  skills  in  the  cognitive  domain  for  one  of  the  most 
essential  objectives  in  the  B/N  training  program,  i.e. , the  recognition  and  assessment  of 
the  pertinent  parameters  affecting  a weapon  delivery  and  the  selection  of  the  correct  response 
for  subsequent  action  based  on  these  parameters. 

The  Procedures  Trainer  (PT)  should  include  the  following  major  features: 

a.  A replica  of  the  B/N  (or  pilot)  disolays  and  controls,  including  any  pilot  displays 
directly  affected  by  B/N  action. 

b.  A minimum  set  of  simulated  CRT  displays  to  reflect  significant  display  responses 
to  B/N  (or  pilot)  actions. 

c.  A capability  to  handle  a repertoire  of  part-  and  full-mission-oriented  procedures. 

d.  A means  to  quickly  set  up  initial  conditions,  hold  action  or  change  runs. 

e.  A training-oriented  interactive  system  consisting  of  an  error  recognition  system, 
response  system,  timers,  counters,  etc. 

f.  Capability  to  randomly  introduce  conditions  requiring  timely  assessment  and  im- 
plementation of  the  most  correct  course  of  subsequent  action. 

g.  Portability  and  easy  setup  using  conventional  power  supplies  and  training  facilities. 

h.  Capability  to  easily  reconfigure  the  trainer  and  its  scenario  sets. 
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i.  Ease  of  maintenance. 

j.  High  reliability 

As  an  example  of  the  technology  that  is  available  for  this  type  of  device,  emergency 
procedure  trainers  currently  being  manufactured  by  Grumman  (generically  known  as  Suit- 
case Emergency  Procedures  Trainers-SCEPTR)  conceptually  incorporate  many  of  the 
features  recommended  for  the  A-6  Procedures  Trainers.  The  SCEPTR  units  are  portable, 
can  be  rapidly  set  up  and  are  capable  of  operation  in  a train  or  quiz  mode.  Train  mode 
enables  the  student  to  attain  proficiency  in  emergency  procedures.  The  quiz  mode  gages  his 
progress. 

Using  the  Tasks  and  SBOs  selected  as  candidate  material  for  a PT  program,  an  exam- 
ination was  made  into  the  possibility  of  expanding  the  basic  SCEPTR  concept  to  include  the 
capability  required  for  a B/N  or  Pilot  PT.  It  has  been  concluded  that  the  basic  concept 
satisfies  the  requirements  of  a Pilot  PT.  In  the  case  of  a Pilot  PT  the  procedural  aspects  of 
operating  the  A/C  are  more  straight-forward  and  can  be  keyed  directly  from  approved  nom- 
inal and  emergency  checklists.  Consequently,  the  current  SCEPTR  design,  including  the 
addition  of  a random  failure  mode,  need  not  be  altered  appreciably.  Less  capacity  would  be 
required  for  sequential  or  alternate  path  routines  and  therefore  considerably  less  non-recur- 
ring programming  time. 

The  major  efforts  involved  in  expanding  the  concept  for  a B/N  PT  concern  the  follow- 
ing points: 

a.  Inclusions  of  Simulated  CRT  Presentations  for  the  VDI,  DY’RI,  and  FLER  Indica- 
tors - A selected  number  of  pertinent  displays  could  be  projected  onto  a screen  and  when 
necessary  could  simulate  motion  with  sufficient  fidelity  to  accomplish  the  overall  learning 
objective. 

b.  Expansion  of  the  Present  (>K  Memory  to  10K  or  Greater  - As  a result  of  expanded 
switch  and  display  requirements  and  the  inclusion  of  sequentially  oriented  nominal  and  off- 
nominal  mission  routines,  storage  tables  would  be  larger  and  the  present  size  of  the  execu- 
tive program  would  have  to  be  significantly  increased  to  accommodate  management  of  the 
required  algorithms. 

c.  Simulation  and  Replication  of  the  Computer  Keyboard  and  Displays  - LEDs  could 
be  used  to  display  data  feedback  for  a selected  number  of  items,  such  as,  the  temporary 
storage  register  display  (in  lieu  of  rotating  drums  or  tape  drives).  Other  data  displays 
could  be  limited  to  predesignated  values  or  covered  by  advisory  displays  to  minimize  storage 
requirements. 

d.  Inclusion  of  A Selectable  Random  Failure  Routine  - A recognition  mode  of  opera- 
tion is  currently  being  designed  for  incorporation  into  a helicopter  SCEPTR.  When  activated, 
a system  configuration  is  displayed  resulting  from  a randomly  injected  failure  condition  at 

a discrete  time.  A display  then  offers  the  student  a choice  of  up  to  eight  diagnoses  :ind  sets 
a routine  to  enable  and  monitor  the  alternate  routine. 

Including  the  expanded  capability  of  the  recommended  A-6  PT,  the  hardware  costs 
would  not  significantly  increase  over  the  hardware  cost  of  the  presently  available  SCEPTR. 
However,  a considerably  higher  nonrecurring  programming  cost  would  be  incurred  for  the 
B/N  PT. 
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Weapon  System  Trainer.  The  Media  Analysis  has  identified  235  SBOs  that  can  best  be 
accomplished  on  a Weapons  System  Trainer  (WST).  Of  these  SBOs,  146  are  related  to  nom- 
inal pilot  and  B/N  tasks  while  the  remaining  89  are  related  to  Airframe  Emergencies  and 
System  Malfunctions. 

An  A-6E  Weapons  System  Trainer,  Device  2F114  (incorporating  the  requirements 
stipulated  in  NTEC  Specification  2221-1133A,  16  Sept.  1974,  Task  4062  and  supplemented  by 
TRAM  characteristics)  will  satisfy  these  SBOs.  These  requirements  call  for  a trainee 
station  (i.e. , simulated  side  by  side  pilot  and  bombardier/navigator  cockpits)  with  its  asso- 
ciated instructor  station  having  input/output  controllers  and  conversion  units.  These  equip- 
ments will  be  activated  by  a general  purpose  digital  computer  system.  A Digital  Radar 
Landmass  Simulation  (DRLS)  System  should  be  provided  with  each  trainer  unit.  The  trainer 
should  be  capable  of  simulating  normal,  emergency  and  degraded  operation  of  the  A-6E  air- 
craft including  aircraft  control,  instrument  procedures,  airframe  system  and  engine  control, 
emergency  procedures  and  all  modes  of  weapons  systems  operation  and  delivery.  Close 
coordination  between  the  pilot  and  B/N  is  provided  during  all  aspects  of  simulated  A-6E 
operation.  The  following  is  a brief  description  of  some  of  the  more  important  aspects  of  the 
A-6E  that  should  be  simulated: 

a.  Aerodynamic  performance  and  flight  characteristics. 

b.  Simulated  motion  of  ownship  with  respect  to  any  geographical  location  from 
sea  level  to  50,000  ft. 

c.  Power  Plant  Systems  - Throttles,  ignition  system,  CSD,  engine  oil  system,  en- 
gine bleed  air  system,  engine  instruments,  engine  fuel  control  system. 

d.  Primary  Aircraft  Systems  - Aircraft  fuel  supply  system,  fuel  quantity  indicating 
system,  hydraulic  power  supply,  system  flight  controls. 

e.  Auxiliary  Aircraft  Systems  - Automatic  flight  control  system,  automatic  carrier 
landing  system,  approach  power  compensator,  speed  brakes,  landing  gear  system, 
nose-tow  catapult  system,  wing  fold  system,  wheel  brake  systemT  arresting  hook 
system. 

f.  Flight  Instruments  and  Indicator  Lights  - Angle  of  attack  system,  pitot  static 
system,  MA-1  compass  system,  RSI,  radar  altimeter  system,  caution,  warning 
and  advisory  lights. 

g.  Canopy  - Canopy  switch,  switch  caution  light,  manual  canopy  handle,  boarding 
ladder  switches,  auxiliary  hydraulic  pumps,  hand  pump,  electric  motor  driven 
pump,  a solenoid  thumper  indicating  successful  canopy  jettison,  canopy  de-fog 
system. 

h.  Lighting  Systems. 

i.  Communication,  Navigation  and  Instrumentation  System  (CNI). 

j.  Attack  Navigation  Systems  - Search  Radar  and  indicators.  Inertial  Navigation 
System  (INS),  Doppler  Navigation  System,  Pedestal  Control  Unit,  Digital  Display 
Unit,  Air  Data  Computer.  The  On-Board  Computer,  AN/ASQ-155,  is  an  actual 
operational  computer. 
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k.  Systems  Displays  - Direct  View  Radar  Indicator  (DVRI),  Pilots  Control  Box  (PCB), 
Vertical  Display  Indicator  (VDI),  Optical  Sight,  Forward  Looking  Infrared  (FLIR) 
Indicator,  Sidewinder,  Shrike  and  ECM  displays  including  aural  tones. 

l.  Weapon  Release  System  - Armament  Control  Unit,  fuze  function  control  panel, 
nuclear  panel,  manual  release  buttons,  reselect  logic,  stores  jettison,  weapon 
mounting  simulation.  Ballistics  Control  Panel,  Detection  and  Ranging  Set  (DRS) 
including  laser  and  Forward  Air  Controller  ranging  and  control  functions. 

m.  Fire  Detection  System. 

n.  In-flight  Refueling  System  - Simulation  limited  to  cockpit  functions. 

o.  Electronic  Countermeasures  (ECM)  - Active  and  passive  ECM  capabilities, 
chaff  dispensing. 

p.  Built-in  Test  (BIT)  Systems. 

There  are  certain  A-6E  capabilities  that  are  not  simulated  in  the  currently  planned 
WST  update.  One  example  is  A-6E  VFR  shore  and  carrier  landings.  If  the  expanded  WST 
had  a visual  land  mass  simulator  and  some  other  minor  refinements,  the  pilot  could  practice 
VFR  operations.  In  addition,  this  modification  would  allow  the  use  of  the  optical  sight  for 
navigation  checks  (COP)  and  visual  attack  modes.  Another  shortcoming  of  the  currently 
planned  WST  is  its  static  FLIR  display.  Many  of  the  tracking  and  attack  procedures  require 
the  use  of  FLIR  imagery  which  does  not  and  will  not  exist  on  the  WST  display. 

Our  analysis  has  shown  that  a considerable  number  of  specific  behavioral  changes 
associated  with  these  visual  operations  would  best  be  achieved  through  training  in  a WST. 

TC-4C  Inflight  Trainer.  As  a result  of  the  media  study,  15  Specific  Behavioral  Objectives 
(SBOs)  have  been  identified  that  can  be  performed  in  a TC-4C  aircraft.  All  of  the  objectives 
are  related  to  navigation  and  system  steering  using  the  Search  Radar  and  the  4 it  Computer. 

The  TC-4C  is  a modified  Gulfstream  that  is  equipped  with  a radome,  and  an  A-fiE  con- 
figured cockpit  in  the  cabin  with  several  B/N  repeater  stations.  It  is  primarily  a B/N 
training  device  used  to  teach  the  techniques  of  navigation,  target  acquisition  and  target 
tracking  under  actual  flying  conditions.  Although  the  introduction  and  some  of  the  practice 
phases  of  many  of  the  SBOs  might  be  satisfied  in  the  WST,  the  TC-4C  provides  valuable 
in-flight  demonstration  of  the  capability  of  the  B/Ns  to  use  the  operational  equipment  under 
actual  flight  conditions.  The  multistation  capability  also  provides  simultaneous  training  of 
2 to  3 B/Ns.  A Grumman  proposal  has  been  submitted  to  the  Navy  to  retrofit  an  existing 
TC-4C  aircraft  with  a Detection  and  Ranging  Set  (DRS).  If  this  proposal  is  accepted  and  the 
retrofit  is  accomplished,  the  target  acquisition  and  tracking  procedures  using  FLIR  imagery 
can  be  practiced  and  demonstrated  under  actual  flying  conditions. 

Paper  Simulation.  Throughout  this  study,  the  Navy  SME 'Pilot  and  B 'N  Instructors  have 
made  repeated  reference  to  something  called  "headwork".  After  much  discussion,  the 
following  operational  definition  of  this  term  was  developed:  "The  ability  to  perform  in  a 
satisfactory  manner  in  situations  for  which  one  has  not  teen  specifically  trained.  " The 
evaluation  of  "headwork"  is  currently  done  in  the  A-6  Readiness  Squadron.  It  is  found  as  a 
line  item  on  many  flight  rating  forms. 
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As  an  aid  in  developing,  recognizing,  and  measuring  "headwork"  in  the  nonflight 
environment  it  is  felt  that  the  category  of  media  called  paper  simulation  could  be  effectively 
utilized.  Paper  simulation  is  the  representation  of  selected  dynamic  characteristics  of  a 
system  through  the  use  of  charts,  tables,  static  photographs,  drawings  and  lists  of 
performance  characteristics  under  specified  conditions.  This  information  is  presented  in 
such  a way  that  the  trainee  can  study  the  initial  performance  of  the  system,  change  inputs  to, 
or  elements  within  the  system  and  note  changes  in  the  performance  of  the  system.  It  should 
be  noted  that  although  paper  simulations  are  not  a part  of  the  current  flight  crew  training 
syllabus,  they  are  being  used  successfully  in  other  areas,  notably  management  training. 

They  have  proven  effective  in  providing  both  practice  and  measurement  of  the  higher  order 
mental  skills  involved  in  planning  and  decision  making. 

It  is  felt  that  simulations  of  this  type  could  be  developed  for  mission  planning  and 
mission  contingencies.  The  learners  would  be  given  certain  information,  some  relevant, 
some  not,  and  asked  to  make  decisions  which  would  affect  the  planning  and  carrying  out  of 
the  mission.  This  could  be  done  on  an  individual  basis  or  in  groups  to  enhance  the  com- 
petitive aspects  of  the  exercise.  At  the  appropriate  point  in  the  exercise,  the  learner  is 
brought  face  to  face  with  the  results  of  his  earlier  decisions.  This  technique  could  be 
effectively  used  in  enhancing  and  measuring  the  decision-making  aspects  of  the  job  of  air- 
crewman  in  areas  such  as  target  and  route  planning,  ordnance  selection,  fuel  planning,  alter- 
nate mission  planning  etc.  It  can  also  be  used  to  introduce  unplanned,  but  "real"  contin- 
gencies to  the  nominal  "simulated"  mission.  The  student's  response  to  unforeseen  changes 
in  weather,  target  priority,  enemy  defenses,  and  A/C  or  systems  operation  can  be  observed 
and  critiqued. 

The  oaper  simulations  can  be  prepared  and  stored  for  use  at  the  instructor's  dis- 
cretion. If  done  realistically,  they  can  add  an  enriching  element  to  the  more  traditional 
syllabus. 


61 


NAVTRAEQUIPCEN  75-C-0099-1 


MEDIA  SURVEY.  As  part  of  the  media  selection  process  a field  site  survey  of  media  currently 
in  VA-42  and  VA-128  inventory  was  conducted.  Selected  media  devices  found  to  be  already 
In  inventory  were  examined  as  to  their  inherent  capabilities  to  satisfy  the  SBO/leaming 
points  associated  with  them.  The  devices  examined  were  those  which  are  for  learner  use, 
such  as  carrels,  video  players,  sound  slide  devices,  etc.  Other  equipment,  such  as  portable 
video  recording  systems,  studio  video  systems,  audio  tape  copiers,  etc.  , were  recognized 
as  production  or  development  devices  and  accordingly  outside  the  considerations  of  being 
appropriate  for  learner  use  and  therefore  not  included  in  the  analysis, 
the  analysis. 

The  survey  revealed  a significant  disparity  in  the  media  resources  of  the  two  training 
squadrons.  VA-128  has  an  extensive  inventory  of  television  equipment  consisting  of  a 
studio  and  portable  recording  system,  including  a special  effects  generator,  studio  type 
video  recorder,  six  color  monitors  (suitable  for  classroom  use)  as  well  as  lenses,  camera 
stands  and  tripods,  lights  and  microphones.  In  addition,  there  are  six  video  cassette 
players,  one  video  cassette  recorder  and  six  T.  V.  receivers  (suitable  for  carrel  use). 
Furthermore,  VA-128  has  12  study  carrels,  six  Synsor  LEM  models  and  six  Howe  T/E-SS 
& T/E-SA  with  six  Howe  rear  screen  projection  units  for  individualized  multimedia  learn- 
ing. The  audio-visual  devices  include  six,  35mm  slide  projectors  and  eight  Telex  sound - 
sync  audio  players,  a Telex  audio  cassette  copier,  two  16mm  projectors,  a stereo  audio 
recorder  and  twelve  headphones.  It  is  apparent  from  this  survey  that  VA-128  is  relatively 
well-equipped  for  meeting  the  media  candidate  requirements  identified  in  this  report. 

Although  VA-128  has  an  extensive  inventory  of  equipment  there  exists,  for  them,  the 
problem  of:  (1)  assembling  and  configuring  the  learning  carrels  for  optimum  utilization  of 
equipment  and  facilities;  and  (2)  the  training  of  selected  instructor  personnel  in  the  use  of 
multimedia  devices. 

VA-42  squadron  has  a severely  limited  media  capability.  Their  inventory  consists  of 
the  following: 

• Three  overhead  projectors 

• Three  slide  projectors 

• One  rear  projection  sound-slide  device 

• One  lfimm  projector  (reel-to-reel) 

• Three  tape  recorders  (not  suitable  for  making  or  reproducing  tapes  for  the  sound- 
slide  device) 

A comparison  of  the  limited  VA-12  inventory  with  the  training  support  requirements 
stated  in  the  report  reveals  the  need  for  the  purchase  of  the  following  equipment: 

• Learning  carrels 

• Sound- slide  devices 

• Video  cassette  players 


• Headphones 
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Except  for  headphones,  specific  equipment  could  be  selected  from  the  data  sheets  con- 
tained in  the  Training  Support  Requirements  document.  (1) 

INSTRUCTIONAL  STRATEGIES  AND  SEQUENCES 

INTRODUCTION.  Lesson  Specifications  forA-OE  (TRAM)  replacement  pilots  and  bombardier/ 
navigators  v re  developed  by  the  Navy/contractor  ISD  Team.  The  primary  data  sources 
used  in  the  preparation  of  these  Lesson  Specifications  were  the  Instructional  Systems  De- 
sign Records,  which  contain  the  Specific  Behavioral  Objectives  and  their  associated  Cri- 
terion Tests,  and  the  Media  Analysis  Tables  from  the  Training  Support  Requirements 
Document.  1 

Following  the  completion  of  the  Media  Analysis,  the  first  step  in  the  process  of  de- 
veloping the  Le.  son  Specifications,  was  to  analyze  the  SBOs  to  determine  those  which  were 
independent  of  each  other,  those  which  were  dependent  and  those  which  had  supportive  re- 
lationships. When  two  SBOs  are  independent,  the  learning  of  one  has  no  effect  on  the 
learning  of  another.  When  two  SBOs  are  dependent,  it  is  necessary  to  learn  one  prior  to 
learning  the  other.  The  third  possibility  exists  when  the  learning  of  one  SBO  supports  or 
facilitates  the  learning  of  another,  but  the  order  in  which  they  are  learned  is  not  important. 
The  learning  in  one  will  transfer  to  the  other  no  matter  which  one  is  learned  first.  The 
outcome  of  this  process  is  a learning  heirarchy  which  forms  the  first  firm  determinant  of 
the  sequence  of  instruction  to  be  followed  in  the  training  program. 

In  actually  performing  this  analysis  it  became  apparent  that  many  of  the  dependent 
relationships  were  already  identified  within  the  terminal-enabling  SBO  structure.  That  is, 
most  terminal  SBOs  had  enabling  objectives  tied  to  them  bv  SBO  number  and  location, 
within  the  tliree  volumes  of  SBOs.  The  learning  of  the  enabling  SBOs  was,  quite  naturally, 
placed  before  the  terminal  SBO.  The  remaining  portion  of  the  task,  that  of  sequencing  the 
terminal  SBOs,  was  found  to  be  more  complex  because  of  the  high  incidence  of  supportive 
and  independent  SBO  relationship. 

The  relative  lack  of  firm  hierarchial  dependent  relationships  among  terminal  SBOs  led 
to  the  consideration  of  instructional  sequencing  based  upon  context:  either  "systems"  or 
"phase  of  flight".  It  was  recognized  that  there  is  an  instructional  advantage  to  introducing 
the  system  e.g. , location  and  operation  of  controls  and  the  necessary  concepts  for  initial 
operation  of  the  system  while  allowing  the  trainee  to  interact  with  the  hardware  as  soon  as 
possible.  That  this  active  participation  must  occur  within  the  correct  context  of  phase  of 
flight  in  order  to  establish  a valid  framework  of  associations  led  to  the  strategy  of  inte- 
grating "systems  knowledge"  with  actual  "systems  operation".  The  use  of  the  Procedures 
Trainers,  the  WST,  and  the  TC-4C  allows  the  application  of  this  instructional  strategy  without 
risking  lives  or  aircraft  in  the  early  learning  stages.  The  task  of  sequencing  the  terminal 
SBOs  became  one  of  integrating  the  procedural  "systems  operation"  objectives  with  the 
"systems  knowledge"  objectives  in  an  order  which  reflects,  as  closely  as  possible,  the 
operational  sequence  of  events  e.g.  , teach  preflight  activities  before  tactics. 

The  Training  Support  Requirements  Document1  lists  seven  learning  principles  or 
guidelines  which  have  been  identified  as  applying  to  the  training  of  A-6E  /TRAM  crewmen. 

The  first  stated  the  desirability  of  providing  early  hands-on  practice  and  has  been  discussed. 


1 J.  Feddem,  et  al..  Report  ISD  (SAT)  0004,  op.  cit.  p.  31. 
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Of  the  remaining  six  guidelines,  four  were  found  to  have  significant  impact  on  the  design  of 
the  Lesson  Specifications.  They  are: 

a.  Task  integrity  and  continuity  should  be  preserved  wherever  appropriate  in  training. 

b.  The  principles  of  mass  versus  distributed  practice  should  be  considered  through- 
out the  program. 

c.  Tasks  requiring  considerable  skill  development  should  be  introduced  and  practiced 
early  in  the  program. 

d.  Maximum  transfer  of  training  skill  and  knowledge  to  the  operational  environment 
can  be  achieved  with  maximum  use  of  "real"  hardware. 

The  first  three  guidelines  are  functions  of  sequencing  and  blocking.  That  is,  success 
or  failure  to  implement  these  guidelines  depends  upon  the  order  in  which  SBOs  are  sequenced 
for  instruction,  the  grouping  of  SBOs  to  be  taught  together,  the  duration  of  instructional 
blocks,  and  the  duration  and  spacing  of  practice  periods.  Success  in  implementing  the 
fourth  guideline  is  determined  by  the  decisions  made  in  assigning  media  to  newly  grouped 
blocks  of  SBOs.  (This  often  involves  compromises  between  the  optimum  media  assigned 
each  individual  SBO  during  the  media  analysis). 

In  addition  to  the  consideration  of  learning  guidelines  during  the  formulation  of  the 
Lesson  Specifications,  there  are  three  operational  characteristics  of  the  SBOs  themselves 
which  can  come  into  play  in  determining  the  durations  and  sequencing  of  instructional  blocks. 
These  characteristics  are: 

• Difficulty  (both  to  perform  the  required  behavior  an<3  to  learn  it) 

• Criticality 

• Frequency  of  Occurrence 

The  amount  of  time  allocated  to  teaching  and  study /practice  of  a given  SBO  is  directly 
related  to  the  degree  of  difficulty  in  performing  or  learning  that  behavior.  The  Task 
Analysis  Record  includes  ratings  of  task  difficulty  along  with  a checklist  of  difficulty  factors 
i.e.  , the  reasons  for  a task  being  difficult,  such  as  - limited  time  or  number  of  cues 
available.  Experience  shows  that  while  the  Task  Analysis  Record  points  to  behaviors  which 
pose  difficulty  to  the  student,  the  judgments  of  the  SMEs  are  invaluable  in  qualifying  these 
difficulties  and  quantifying  the  time  generally  required  for  instruction  and  practice/study. 

Criticality  of  an  objective  influences  the  training  time  in  that  if  an  objective  e.g.  , 
engine  failure  emergency  procedure,  has  high  criticality  with  respect  to  crew  safety  or 
mission  success,  training  time  must  be  sufficient  to  ensure  that  the  trainee  can  accomplish 
the  objective.  If  the  objective  is  not  crucial,  then  less  training  time  is  appropriate.  The 
Task  Analysis  Record  contains  entry  locations  for  criticality  which  were  helpful  in  devel- 
oping the  Lesson  Specifications.  These  entries  cover  the  range  from  no  criticality,  through 
Degraded  Mission,  to  Crash/Death. 

The  third  operational  characteristic  which  was  particularly  considered  in  developing 
the  specifications  was  that  of  frequency  of  occurrence.  If  the  frequency  of  the  event  is 
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"high"  the  behavior  required  to  handle  tlie  event  must  be  trained  and  the  time  allocated  for 
training  that  behavior  must  be  ample.  The  criteria  of  "high"  frequency  is  not  clearly  de- 
fined, although  the  Task  Analysis  Record  addresses  it  under  the  heading  of  Specific  Train- 
ing Factors  with  entries  such  as  "Normal  Recurring"  and  "Requires  Frequent  Practice". 
Once  again,  the  judgement  of  operational  personnel,  in  the  person  of  Navy  SMEs,  was 
required. 

PROCESS.  1'he  task  ot  analyzing  the  SBOs  in  preparation  for  sequencing  and  blocking  began 
with  the  physical  separation  of  the  SBOs  from  the  three  volumes  of  SBO  Statements1 
and  their  rearrangement  into  two  groups,  one  for  the  Pilot  and  one  for  the  Bombardier/ 
Navigator.  The  determinant  factor  for  this  division  was  the  "Task  Responsibility"  box  of 
the  Instructional  Systems  Design  Record  (Fig.  5).  Tasks  designated  for  only  the  Pilot  or 
B/N  were  placed  in  their  respective  groups,  while  tasks  identified  as  "both",  "shared", 
or  "coordinated"  were  placed  in  both  groups.  This  division  was  necessary  because  the 
Pilot  and  B N documents  contain  SBOs  describing  behaviors  performed  by  the  other  crew- 
man. This  is  an  evolution  of  the  "Mission  Scenario"  format  of  the  Task  Listings,  i.e.,  all 
pre-flight,  takeoff  and  landing  related  tasks  are  in  the  Pilot  volume,  all  tactics  and  weapons 
delivery  tasks  are  in  the  B/N  book.  With  all  SBOs  now  sorted  by  crewman  responsibility, 
individual  contractor  ISD  team  members  were  assigned  to  each  of  the  two  sets  of  SBOs  and 
proceeded  with  the  processes  of  developing  Lesson  Specifications  for  each  crewmember. 

The  task  of  analyzing  the  SBOs  prior  to  sequencing  and  blocking  was  handled  slightly 
differently  by  the  ISD  team  members.  In  the  case  of  B/N  SBOs  it  was  found  that  system/ 
subsystem  associations  predominated  and  therefore  a grouping  of  SBOs  by  system/subsystem 
e.g.  , Air  Data  Computer,  Detection  and  Ranging  Set,  etc.  was  found  to' be  useful  and  ap- 
propriate. Informal  summary  worksheets  were  used  to  sort  the  SBOs  by  systems/sub- 
system. These  worksheets  also  included  checkoff  boxes  for  a variety  of  operational  and 
training  characteristics,  such  as,  difficulty,  criticality,  and  frequency.  The  data  sources 
for  these  worksheets  were:  the  Task  Analysis  Records,  the  Task  Listings,  and  the  SBOs 
themselves.  The  outcome  of  the  initial  use  of  these  worksheets  was  the  division  of  B/N 
SBOs  into  seven  groups  of  activities,  i.e..  Familiarization,  System/Subsystem  Design, 
Theory  and  Application,  Airframe  Emergencies  and  System  Malfunctions,  Navigation  Theory 
and  Application,  Attack  Theory  and  Application,  Air/Combat  Theory  and  Application,  and 
Takeoff  and  Landing  Checks. 

The  pilot  SBOs  were  found  to  be  less  associated  with  systems/subsystems  than  with 
phase  of  flight  and  phase  of  mission.  For  this  reason  it  was  deemed  appropriate  to  an- 
notate the  SBO  Record  Sheets  themselves  with  the  operational  and  training  characteristics. 
This  data  was  taken  directly  from  the  Task  Analysis  Records  which  in  themselves  are 
mission/phase  of  flight  oriented.  The  group  of  Pilot  SBOs  was  then  physically  sub-divided 
into  six  groups  with  the  following  titles:  Familiarization,  Visual  Weapons  Delivery,  Tactics, 
Navigation,  System  Weapons  Delivery,  and  Carrier  Qualifications.  These  first  groupings 
of  both  pilot  and  B/N  SBOs  were  designated  as  Training  Phases.  The  phases  were  then 
further  divided  into  units  consisting  of  either  systems  related  or  phase  of  flight  related  SBOs. 
The  phases  and  units  were  in  themselves  blocks  of  instructional  requirements,  with  the 
ordering  of  units  within  the  phases  suggesting  the  first  sequencing  of  instruction.  The 
educational  specialists  performing  this  first  and  second  level  blocking  and  sequencing  were 
influenced  by  the  learning  guidelines  as  well  as  the  analytical  data  relative  to  each  SBO. 


1 Grumman  Reports  lSI)  (SAT)  0009-4,  5 and  0,  op.  cit.  p.  45. 
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With  the  SB  Os  divided  into  phases  and  units,  the  next  step  was  a review  by  the  SMEs 
from  VA-42  and  VA-128.  This  took  place  during  a three-day  working  conference  at  NAS 
Oceana.  The  major  activity  at  these  meetings  was  an  in-depth  examination  of  the  content  of 
the  phases  and  units  in  light  of  the  practical  experience  of  tlie  SMEs  as  I>ilots  and  B/Ns, 
and  as  instructors  in  an  existing  A-fi  Readiness  training  program.  Hie  SMEs  were  able  to 
point  out  a number  of  practical  or  operational  considerations  which  suggested  changes  in  the 
phases  and  units,  e.g.  , Navy  minimum  flight  hour  requirements  prior  to  tanker  operations 
and  carrier  qualification.  They  also  assisted  in  the  identification  of  specific  lessons  with- 
in the  units  and  provided  estimates  of  time  for  Initial  Learning,  Practice /Study,  and  De- 
monstration/Test based  on  their  experience  as  students  and  instructors.  Two  significant 
observations  were  made  by  the  SMEs  during  these  meetings: 

a.  The  bulk  of  the  aircraft  systems  training  for  the  current  A-6  Readiness  Training 
program  is  done  by  the  Naval  Air  Maintenance  Training  Group  in  highly  concentrated  form 
prior  to  any  Readiness  squadron  hands-on  training.  It  was  felt  that  much  of  this  training 
was  not  retained  because  it  was  not  associated  with  operational  activities.  This  was  es- 
pecially true  in  the  case  of  Replacement  B/Ns,  who  do  not  even  have  the  aircraft  operational 
experience  of  the  Replacement  Pilots,  at  least  18  months  experience  in  the  Training  Com- 
mand. 


b.  The  current  Readiness  Squadron  training  program  includes  replacement  B/Ns  in 
basically  the  same  Familiarization  Phase  as  the  Replacement  Pilots  at  the  beginning  of  the 
program.  The  goal  is  to  prepare  the  RBNs  to  function  as  copilots.  Again,  the  RBNs  do  not 
have  the  same  entry  level  skills  and  knowledges  as  the  RPs  and  much  of  the  training  is  not 
retained.  The  SMEs  were  unanimous  in  their  opinion  that  the  bulk  of  the  real  learning  on 
the  part  of  BNs,  in  the  areas  of  aircraft  systems  operation  and  copilot  activities,  took  place 
in  the  operational  squadrons. 

Based  on  these  considerations,  the  following  instructional  strategies  were  recommend- 
ed: 

a.  The  aircraft  systems  training  currently  being  done  by  the  Maintenance  Training 
Group,  should  be  done  in  the  Readiness  Training  program  as  part  of  the  Famili- 
arization Phase.  Furthermore,  (he  content  of  that  training  should  reflect  an  op- 
erational rather  than  a maintenance  orientation. 

b.  The  copilot  training  given  the  RBNs,  including  the  relevant  portions  of  the  aircraft 
systems  training,  should  be  placed  late  in  the  Readiness  Squadron  RBN  curriculum 
to  allow  greater  association  with  actual  flight  operations. 

At  the  conclusion  of  the  meetings  at  NAS  Oceana,  the  contractor  ISD  team  proceeded 
with  the  preparation  of  Lesson  Specifications.  This  process  involved  the  following 
activities: 

a.  The  units  of  instruction  were  further  subdivided  into  lessons  with  SBOs  identified 
and  assigned  to  each.  The  sequence  of  lessons  within  the  units,  and  units  within  the  phases 
were  finalized,  based  on  their  hierarchical  interrelationships. 

b.  The  phases  and  units  of  instruction  were  adjusted,  based  on  the  discussions  with  the 
SMEs.  The  seven  preliminary  B N Phases  were  consolidated  ini"  four  phases  made  up  of 

a total  of  10  units.  It  was  decided  that  the  familiarization  and  carrier  qualification  phases 
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of  the  B/N's  program  would  consist  of  those  portions  (lessons)  of  the  pilot  program  which 
contained  B/N  activities.  The  Replacement  Pilot's  training  program  would  contain  six 
Phases  containing  a total  of  12  units.  The  Navigation  and  Target  Acquisition  and  Weapon 
Delivery  Phases  of  the  Pilot's  program  would  be  made  up  of  the  lessons  from  the  B/Ns 
program  which  contained  Pilot  activities. 

c.  The  media  to  be  used  in  each  lesson  was  determined  based  on  the  recommendations 
made  in  the  media  analysis.  As  previously  noted,  this  process  involved  the  making  of 
certain  compromises  with  the  optimum  media  previously  assigned  to  individual  SBOs. 

That  is,  the  total  group  of  SBOs  in  a lesson  could  include  six  different  media  recommenda- 
tions from  the  Media  Analysis.  Practicality  requires  that  this  number  be  reduced  by 
selecting  a smaller  group  of  media  may  not  be  optimum  for  certain  of  the  SBOs  in 
the  lesson,  e.  g. , a sound-slide  program  and  printed  handouts  are  the  designated  media  for 
a lesson  satisfying  a set  of  SBOs  which  originally  called  for  overhead  transparencies,  wall- 
charts,  sound-slide  program,  printed  handouts,  video  tape  and  ltimm  film. 

d.  The  lessons  were  (where  it  was  appropriate)  further  divided  into  "sessions"  based 
upon  hierarchical  and  media  considerations. 

OUTCOME.  As  a result  of  the  foregoing  activities,  specifications1  for  55  lessons  were 
developed  with  19  designated  for  the  pilot  only,  17  for  the  B/N  only,  and  19  common  lessons 
for  pilot  and  B/N.  A summary  of  lessons  is  presented  in  Appendix  A along  with  their 
estimated  times  for  initial  instruction  and  projected  hours  of  simulator  utilization.  The 
lessons  were  documented  on  lesson  Structure  sheets  (see  Fig.  7)  under  the  three  headings 
of  (1)  Initial  Learning;  (2)  Practice;  and  (3)  Demonstration /Test.  The  Initial  Learning  boxes 
contain  the  SBO  numbers  that  are  being  addressed  in  the  lesson/session,  a statement  briefly 
describing  the  instructional  outcome  of  the  lesson/session,  and  a listing  of  recommended 
media  for  the  instruction.  In  some  cases,  both  primary  and  alternate  media  are  specified. 
The  practice  boxes  include  a statement  on  the  recommended  method  of  practice/study,  and 
a listing  of  primary  and  alternate  media  to  be  used.  The  Demonstration/Test  boxes  specify 
the  media  and  methods  to  be  used  in  either  Demonstrations,  (for  psychomotor  activities),  or 
Tests  (for  cognitive  activities). 

In  addition  to  the  Lesson  Structure  Sheets  each  Lesson  Specification  includes  the 
following  information: 

a.  Lesson  Number  and  Title  - The  lesson  number  consists  of  an  alpha-numeric  de- 
signator where  the  letters  P,  B or  P and  B indicate  the  crewmember  for  whom  the  lesson  is 
intended.  P indicates  Pilot,  B indicates  Bombardier  /Navigator  and  P and  B designates  the 
lesson  for  both.  The  first  number  in  the  series  designates  the  phase  of  training,  the  second 
indicates  the  unit  within  the  phase,  and  the  third  specifies  the  lesson  within  the  unit.  The 
title  for  each  phase,  unit  and  lesson  is  also  provided. 

b.  Specific  Behavioral  Objective  (No.  & Statement)  - The  left-hand  column  consists 
of  a listing  of  the  SBO  numbers  that  are  being  addressed  in  the  lesson.  A given  SBO  some- 
times appears  in  more  than  one  lesson.  This  is  a function  of  phase  of  flight  and  system  con- 
siderations. 
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Next  to  each  SBO  number  is  the  actual  behavioral  statement,  just  as  it  appears  on  the  In- 
structional System  Design  Record.  During  the  process  of  preparing  Lesson  Specifications, 
additional  enabling  SBOs  were  identified  for  the  pilot  syllabus.  They  are  listed  in  the  Lesson 
Specification  Document. 

c.  Media  - The  media  to  be  utilized  in  each  lesson,  and  session  within  the  lesson,  are 
designated  under  the  heading  of:  Initial  Learning,  Practice  and  Demonstration  Test.  In 
many  cases  there  is  a preferred  set  of  media  designated  by  the  word  PRIM,  and  a second 
choice  indicated  by  the  term  ALT. 

d.  Lesson  Duration  - The  section  consists  of  an  estimate  of  the  time  required  for  the 
replacement  pilot  or  B/N  to  fulfill  the  three  aspects  of  behaviorially  based  learning,  i.e. , 
Initial  Instruction,  Practice,  and  Demonstration/Test.  The  term  "Practice"  includes  the 
activity  of  studying  where  it  is  appropriate. 

e.  References  - This  section  consists  of  a listing  of  the  primary  resource  documents 
to  be  used  in  the  formulation  of  the  detailed  lesson  material.  It  is  anticipated  that  further 
references  will  be  identified  as  lesson  development  continues. 

At  the  time  of  this  writing,  comments  from  the  SMEs  relative  to  their  review  of  the 
Lesson  Specifications  have  not  as  yet  been  received.  It  is  assumed  that  any  changes  based 
on  that  review  will  be  discussed  and  implemented  by  the  Navy/contractor  ISD  Team  prior 
to  any  media  production. 
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SECTION  IV 
COST  ANALYSIS 


CONTRACTOR  AND  NAVY  MANHOURS 

Table  7 is  a comparison  of  Contractor  work  effort,  by  task  and  skill,  with  the  pro- 
posed program  schedule  and  the  estimated  actual  level  of  effort  required  to  complete 
Phase  I.  The  effort  is  expressed  in  manmonths. 

The  NAVY  effort  is  not  available  by  task,  but  has  been  estimated  by  the  SMEs  to 
total  5.5  manmonths  for  VA-42  and  2.25  manmonths  for  VA-128  for  the  entire  Phase  I 
effort. 

ADMINISTRATIVE  AND  SUPPORT  COSTS 

Clerical  and  administrative  effort  is  an  element  of  overhead  and,  as  such,  was  not 
separately  identified  in  the  proposed  program  schedule.  It  is  estimated  that  15  manmonths 
of  this  effort  were  expended  on  Phase  I. 

OTHER  COSTS 

In  addition  to  the  direct  and  indirect  labor  costs  noted  above,  approximately  $27,500 
was  identified  for  data  computing  services,  and  $4,000  was  allocated  for  travel.  Both 
these  amounts  were  expended. 
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TABLE  7.  COSTS  - CONTRACTOR  WORK  EFFORT 


Task 

Proposed  Work 
Plan  (in  M/M) 

Estimated  Actuals 
(in  M/M) 

• Work  Plan  & Quarterly  Reports 

Instructional  Psychologists 

0.5 

0.5 

Educational  Specialists 

0.5 

0.5 

• Task  Analysis 

Instructional  Psychologists 

4.5 

9 

Educational  Specialists 

3.5 

6 

Flight  Crew 

3 

6 

• SBOs 

Instructional  Psychologists 

2 

6 

Educational  Specialists 

9 

11.5 

Flight  Crew 

0.5 

0 

• Media  Analysis 

Instructional  Psychologists 

0 

4 

Educational  Specialists 

2 

4 

• Lesson  Specifications 

Instructional  Psychologists 

2 

2 

Educational  Specialists 

10 

9 

Flight  Crew 

0.25 

0 

• Final  Report  (Preliminary) 

Instructional  Psychologists 

2 

4 

Educational  Specialists 

2 

3 

Flight  Crew 

0.25 

0 

• Final  Report  (Final) 

Instructional  Psychologists 

0.5 

0.5 

Educational  Specialists 

1 

1 

TOTAL  CONTRACTOR  EFFORT 

43.5 

f>7  M/M 

TOTAL  SME  EFFORT 

N/A 

7.  75  M/M 

L, 
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SECTION  V 


CONCLUSIONS 


The  conclusions  found  in  this  section  are  of  various  kinds.  Some  relate  directly  to 
the  aircrew  training  program,  some  to  the  ISD  process,  still  others  have  elements  which 
relate  to  both. 

a.  Operational  oriented  SMEs,  such  as  the  pilot  and  B/Ns  who  participated  in  this 
study  are  not  effectively  utilized  if  nerforming  lengthy,  highly  analytical  ISD 
tasks,  such  as  writing  SBOs.  They  are  more  efficiently  utilized  as  sources  of 
technical  and  operational  expertise.  Our  experience  in  this  study  confirmed  the 
efficacy  of  utilizing  SME  in  this  role. 

b.  In  the  conduct  of  the  ISD  process  there  is  an  advantage  in  separating  the  Criterion 
Test  Statement  from  its  associated  Behavioral  Statement.  By  so  doing,  the 
Behavioral  Statement  retains  its  relationship  to  the  operational  condition  and 
standards,  while  the  Criterion  Test  Statement  can  be  adjusted  to  accomodate 
learning  conditions  and  standards  where  they  differ  from  the  operational  environ- 
ment. 

c.  Utilization  of  SBOs  in  the  development  of  a Flight  Crew  Training  curriculum 
should  increase  the  instructors’  level  of  confidence  in  their  student  evaluation 
techniques  since  the  performance  measurement  tests  are  constructed  using  the 
SBO  Criterion  Test  Statement. 

d.  The  affective  domain  can  be  excluded  from  the  Taxonomy  of  Training  Objectives 
since  the  Readiness  Squadron  student  population  is  carefully  selected,  highly 
motivated  and  cautiously  screened.  The  validity  of  this  conclusion  has  yet  to 
be  tested. 

e.  A significant  number  of  training  requirements  would  best  be  satisfied  using 
Pilot  and  B/N  Procedures  Trainers,  a TC-4C  (TRAM  configured),  and  a WST 
with  visual  capability. 

f.  Sound  slide  devices  and  carrels  as  well  as  video  tape  equipment  are  important 
elements  in  the  media  recommendations.  Initial  analysis  of  existing  inventories 
in  VA-42  and  VA-128  indicates  some  additional  procurements  are  required. 

g.  Much  of  the  current  A-6  aircraft  systems  training  (which  is  being  conducted  by 
the  Naval  Air  Maintenance  Training  Group  at  the  onset  of  the  Readiness  Squadron 
program)  is  not  being  retained,  especially  by  Replacement  B/Ns. 

h.  Replacement  B/Ns  do  not  have  the  necessary  entry  level  skills  and  knowledges 
relative  to  aircraft  operations,  to  support  copilot  training  early  in  the  Readiness 
Squadron  curriculum. 

i.  Electronic  data  processing  is  highly  recommended  for  ISD  processes  involving  a 
substantial  number  of  job  descriptions,  e.g. , analysis  of  shipboard  operations. 
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SECTION  VI 


RECOMMENDATIONS 


This  section  presents  the  study  recommendations,  which  include  the  following: 

a.  Further  analysis  and  development  of  quantitative  standards  for  pilot  and  B/N 
activities  is  recommended. 

b.  In  future  programs  to  develop  Readiness  Squadron  training  curricula  (for  other 
aircraft  models),  the  Training  Command  terminal  behaviors  should  be  examined 
and  compared  with  the  operational  tasks  before  SBOs  are  generated.  If  this 
information  is  not  available,  it  could  be  prepared  as  an  outcome  of  an  ISD  study 
of  the  Training  Command. 

c.  SMEs  should  not  be  compelled  to  function  as  training  analysts,  e.g. , writing 
SBOs,  performing  media  analysis.  The  analytical  portions  of  the  ISD  process 
should  be  left  to  qualified  Educational  Specialists. 

d.  Behavioral  Objectives  and  their  associated  Criterion  Tests  should  be  separate 
statements,  unless  the  conditions  and  standards  for  the  operational  environment 
and  the  training  situations  are  identical. 

e.  An  implementation  program  (Phase  II)  should  be  conducted  to  verify  that  there  is, 
indeed,  an  increased  level  of  instructor  confidence  in  student  evaluation  techniques 
based  on  the  use  of  SBO  derived  Criterion  Test  Statements. 

f.  The  decision  to  exclude  the  affective  domain  from  formal  consideration  during  tbe 
development  of  the  curriculum  should  be  examined  during  the  validation  portion 
of  an  implementation  program. 

g.  It  is  recommended  that  the  necessary  action  be  initiated  to  procure  pilot  and  B/N 
Procedures  Trainers,  TRAM  configured  TC-4C,  and  a visual  capability  for  the 
WST. 

h.  A Phase  rl  program  should  include  the  procurement  of  necessary  sound-slide 
devices,  learning  carrels  and  video  tape  equipment. 

i.  The  airciaft  systems  training  currently  being  done  by  the  Maintenance  Training 
Group,  should  be  done  by  the  Readiness  Squadron  as  part  of  the  Familiarization 
Phase,  ’’he  content  of  that  training  should  reflect  an  operational  rather  than  a 
maintenance  orientation. 

j.  The  copilot  training  given  the  RBNs,  including  the  relevant  portions  of  the  air- 
craft systems  training,  should  be  placed  later  in  the  curriculum. 

k.  Consideration  should  be  given  to  completion  of  computer-based  data  management 
systems  to  permit  easy  updating  of  changes  in  ISD  based  training  program. 
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LIST  OF  ACRONYMS 


A/C 

Aircraft 

ACLS 

All-Weather  Carrier  Landing  System 

ACU 

Armament  Control  Unit 

AFCS 

Automatic  Flight  Control  System 

AFM 

Air  Force  Manual 

AFP 

Air  Force  Pamphlet 

AGL 

Above  Ground  Level 

AMTI 

Airborne  Moving  Target  Indication 

ATC 

Air  Training  Command 

B/'N 

Bombardier/Navigator 

CAINS 

Carrier  Aircraft  Inertial  Navigation  System 

CAS 

Close  Air  Support 

CE 

Circular  Error 

CEP 

Circular  Error  Pattern 

CNI 

Communications  Navigation  Instrumentation 

CPU 

Central  Processor  Unit 

CRT 

Cathode  Ray  Tube 

CT 

Criterion  Test 

DR 

Dead  Reckoning 

DRLS 

Digital  Radar  Landmass  Simulation 

DRS 

Detecting  and  Ranging  Set 

D/T 

Demonstration/Test  (Phase) 

DVRI 

Direct  View  Radar  Indicator 

ECM 

Electronic  Counter  Measures 

EO 

Enabling  Objective 

EW 

Electronic  Warfare 

FAC 

Forward  Air  Controller 

FAM 

Familiarization 

FCLP 

Field  Carrier  Landing  Practice 

FLIR 

Forward  Looking  Infrared 

GAC 

Grumman  Aerospace  Corporation 

IBM 

Instructor  Bombardier 

87 


NAVTRAEQUIPCEN  75-C-0099-1 
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IFR 

Instrument  Flight  Requirements 

IL 

Initial  Learning  (Phase) 

INS 

Inertial  Navigation  System 

I/O 

Input/Output 

IP 

Instructor  Pilot 

IR 

Infrared 

ISD 

Instructional  Systems  Development 

KTs 

Knots 

MSL 

Mean  Sea  Level 

NAS 

Naval  Air  Station 

NFO 

Naval  Flying  Officer 

NTEC 

Naval  Training  Equipment  Center 

P 

Practice  (Phase) 

PCB 

Pilots  Control  Box 

PT 

Procedures  Trainer 

RBN 

Replacement  Bombardier/ Navigator 

RTI 

Radio  Target  Identification 

RP 

Replacement  Pilot 

SAT 

Systems  Approach  to  Training 

SBO 

Specific  Behavioral  Objectives 

SCEPTR 

Suitcase  Emergency  Procedures  Trainer 

SERE 

Survival,  Evasion,  Resistance,  and  Escape 

SME 

Subject  Matter  Experts 

TAC 

Tactical 

TAR 

Task  Analysis  Record 

TO 

Terminal  Objective 

TRADOC 

U.S.  Training  and  Doctrine  Command 

TRAM 

Target  Recognition  Attack  Multisensor 

VDI 

Vertical  Direction  Indicator 

VFR 

Visual  night  Requirements 

VIS  WE  PS 

Visual  Weapons 

WST 

Weapons  Systems  Trainer 
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